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EXECUTIVE  SUMMARY 


Current  FAA  automation  systems  use  reports  from  a  single  beacon  sensor  to  determine  an 
aircraft’s  position,  altitude,  and  identification.  In  en-route  automation,  should  reports  exist  from 
overlapping  sensors,  one  sensor  is  chosen  as  the  preferred  sensor  and  the  others  are  assigned 
backup  supplementary  roles;  these  assignments  are  pre-defined  for  each  sector  of  a  geographic 
grid  termed  “radar  sort  boxes”.  Data  from  supplemental  sensors  are  only  referenced  when  the 
preferred  sensor  reports  are  missing.  This  system  of  operation  is  named  mosaicing.  While 
mosaicing  does  accomplish  integration  of  surveillance  data  from  multiple  sensors,  it  has 
significant  limitations.  For  example,  differences  between  position  reports  for  the  same  aircraft 
from  different  sensors  normally  exist  due  to  sensor  errors,  including  biases.  These  errors 
produce  significant  “hops”  or  “stitching”  in  the  aircraft  positions  when  the  preferred  sensor 
changes  or  when  the  supplemental  sensor  is  used.  The  presence  of  these  phenomena  cause 
controllers  to  employ  increased  separation  standards  in  affected  regions,  significantly  reducing 
the  effective  airspace  capacity. 

In  recent  years,  with  the  advent  of  faster  computers,  new  techniques  for  multi-sensor  surveillance 
integration  have  been  developed.  While  the  techniques  vary  with  the  individual  developer,  they 
all  combine  surveillance  reports  from  multiple  sensors  into  a  single  track  for  each  aircraft.  These 
techniques  are  collectively  referred  to  as  multi-sensor  “fusion”. 

Lincoln  Laboratory  was  tasked  by  the  FAA  to  measure  the  performance  of  a  representative 
sample  of  current  commercial  off-the-shelf  (COTS)  fusion  trackers.  This  effort  included 
cataloging  the  companies  that  have  available  ATC  fusion  trackers,  acquiring  executable  tracker 
images  from  as  many  as  possible  of  these  companies,  preparing  test  data  sets  (recorded  and 
simulated)  to  input  to  these  trackers,  running  the  commercial  tracker  code  on  the  test  sets,  and 
evaluating  the  performance  achieved. 

The  fusion  evaluation  effort  described  in  this  report  made  use  of  four  COTS  trackers: 

1 .  Lockheed-Martin  (Minnesota)  -  MicroEARTS 

2.  Telephonies  -  ATC  tracker  used  abroad 

3.  Eurocontrol  -  operational  ATC  fusion  tracker  (ART AS)  used  in  Europe 

4.  Raytheon  -  STARS  tracker 

As  expected,  since  adaptation  to  specific  sensor  characteristics  is  critical  to  proper  fusion 
performance,  all  of  the  COTS  trackers  required  preparation  by  the  manufacturers  before  they 
could  be  applied  to  our  ATC  databases. 

The  position  and  velocity  information  contained  in  the  tracker  output  reports  were  analyzed  in 
this  study.  Some  of  the  COTS  products  have  additional  fields  internal  to  the  tracker  that  may 
influence  the  information  passed  to  automation  safety  functions. 

Analysis  of  a  fusion  tracker  measured  its  performance  in  the  following  categories: 

1 .  General  accuracy  —  position  and  velocity  in  straight  flight 

2.  How  quickly  it  responds  to  turns  -  and  accuracy  of  turn  data 
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3 .  How  long  it  takes  to  initiate  tracks  -  and  number  of  false  fliarmg 

4.  How  quickly  it  recovers  from  erroneous  correlations 

5.  Its  ability  to  handle  system  biases 

The  analysis  was  made  using  both  recorded  and  simulated  inputs,  and  with  both  single  sensor 
and  multiple  sensor  databases  for  comparison.  The  recorded  data  runs  indicate  qualitatively,  in 
an  overall  average  sense,  how  well  the  trackers  perform  on  real  ATC  data,  and  show  how  much 
benefit  may  accrue  with  their  introduction.  The  simulation  runs  permit  detailed  quantitative 
measures  of  each  aspect  of  tracker  performance,  since  ground  truth  is  known  and  specific 
situations  can  be  generated. 

The  Lincoln  simulation  generates  data  from  a  superset  of  the  sensors  for  which  recorded  data 
was  collected,  namely  all  enroute  sensors  currently  in  the  New  England  region,  plus  all  terminal 
sensors  at  New  England  airports,  plus  the  production  Mode  S  sensor  operated  at  Lincoln 
Laboratory  (hereafter  denoted  as  SURVSEF).  The  characteristics  of  the  simulated  data  were 
matched  to  those  measured  in  the  recorded  data.  For  each  sensor,  both  a  non-Mode  S  CD2 
format  version  and  a  Mode  S  advanced  format  version  are  defined,  so  that  tests  dependent  upon 
report  data  accuracy  can  be  performed. 

The  basic  simulation  data  set  has  200  aircraft,  each  starting  with  200  seconds  of  straight  constant 
speed  flight,  followed  by  30  seconds  of  standard-rate  turn  3°/sec  turning  right  flight,  followed  by 
another  1 00  seconds  of  straight  flight.  This  database  has  been  used  for  absolute  and  comparison 
performance  tests  for  straight  and  turning  flight  and  the  transitions  between  them. 

This  data  set  was  run  through  each  commercial  fusion  tracker  for  the  following  set  of  sensor 
cases: 

1.  SURVSEF  only  (4.8  second  scan) 

2.  mosaicing  using  all  enroute  sensors  (each  12.0  second  scan) 

3.  fusion  using  all  enroute  sensors 

Each  case  was  run  for  both  CD2  and  Mode  S  quality  reports  as  subcases  to  test  the  effects  of  data 
quality.  The  input  data  in  each  case  was  imbiased;  the  effects  of  data  biases  were  addressed 
separately. 

For  each  case  run,  the  analysis  program  was  used  to  generate  the  absolute  track  angle,  ground 
speed,  and  position  errors  for  each  tracked  output  report.  The  following  characteristics  were  then 
observed: 

1 .  The  minimum  time  needed  to  initiate  tracks. 

2.  The  time  required  for  the  tracker  to  settle  down  for  straight  flight. 

3.  The  performance  during  a  turn  —  especially  how  long  it  took  for  the  turn  to  be  detected. 

4.  The  time  required  for  the  tracker  to  re-establish  straight  flight  steady  state  after  the 
completion  of  the  turn. 
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Since  system  biases  play  a  key  role  in  a  multi-sensor  fusion  system,  each  COTS  tracker  was 
tested  to  determine  its  ability  to  compute  and  adapt  in  real  time  to  their  existence.  The  method  of 
testing  this  performance  characteristic  was  to  play  a  data  scenario  in  which  sensor  biases  were 
present  through  the  trackers  twice:  once  at  the  beginning  of  the  run,  and  then  again  after  an  hour 
of  high-density  aircraft  data.  The  results  indicate  that  all  but  one  of  the  COTS  trackers  have 
algorithms  that  permit  the  tracker  to  adjust  over  time  for  the  existence  of  biases.  We  have  been 
informed  that  the  other  COTS  tracker  has  a  bias  algorithm  that  is  not  yet  part  of  the  automated 
system. 

A  random  scenario  that  mixes  straight,  accelerating,  turning,  and  combination  flight  segments  for 
each  track  was  also  used  to  test  each  COTS  tracker.  The  analysis  computed  the  track  angle, 
ground  speed,  and  positional  errors  for  each  segment  type  as  a  function  of  time  in  the  segment, 
as  well  as  the  time  required  to  recover  from  each  type  of  maneuver  and  return  to  normal  error 
conditions.  This  test  also  included  as  a  benchmark  of  comparison  a  Lincoln  developed  non- 
Kalman  routine  that  estimates  velocity,  but  not  position,  for  each  output  report  when  fed  multi¬ 
sensor  data,  and  thus  can  serve  as  an  adjunct  to  a  full  tracker. 

Dining  the  various  fusion  tests,  it  became  clear  that  the  performance  in  all  cases  was 
significantly  improved  when  Mode  S  data  quality  (such  as  with  ASTERIX)  was  assumed,  as 
opposed  to  the  CD2  data  format  and  accuracy.  A  particularly  important  consequence  of  this 
improvement  was  that  the  significant  number  of  track  drops  noted  in  aircraft  turns  with  CD2  data 
was  virtually  completely  eliminated  when  Mode  S/MSSR  quality  data  was  input  instead  to  the 
trackers. 

The  principal  conclusion  of  this  study  is  that  technology  is  on  the  shelf  (COTS)  to  achieve  some, 
but  not  all,  of  the  surveillance  improvements  that  have  been  promised  with  sensor  fusion 
algorithms. 

The  COTS  products  are  able  to  receive  inputs  from  typical  FAA  radars  and  produce  output 
tracks  that  are  as  good  as  or  better  than  those  produced  by  existing  FAA  automation  systems.  In 
particular,  the  COTS  products  eliminate  the  track  data  “hops”  and  “stitching”  that  are 
characteristic  of  existing  mosaic  systems.  This  could  produce  substantial  benefits  by  opening  up 
the  prospect  of  allowing  controllers  to  use  the  same  separation  standard  permitted  for  single 
sensor  operation  when  multiple  sensors  are  in  use  in  a  given  sector.  The  COTS  products  are 
also  robust  to  the  presence  of  “outlier”  data  and  the  failure  of  a  single  sensor.  In  addition,  it  may 
be  true  that  the  use  of  a  Kalman  filter  by  itself  in  these  COTS  products  in  either  mosaic  or  fusion 
systems  may  lead  to  significant  surveillance  improvements  over  the  current  alpha-beta 
smoothing  (which  was  not  tested). 

The  results  indicated  that  the  COTS  products  do  not  produce  significantly  better  estimates  of 
position  and  velocity  in  the  fusion  mode  than  in  the  mosaic  mode,  either  in  straight  line  or 
turning  flight.  Intuitively,  one  would  have  expected  the  additional  data  and  the  associated  higher 
update  rate  would  have  contributed  to  improved  performance  in  these  areas.  Improved  state 
estimation  might  produce  benefits  by  allowing  decision  support  tools  (e.g.,  CTAS,  etc.)  to  better 
predict  aircraft  trajectories.  At  the  present  time,  however,  FAA  requirements  for  aircraft  state 
estimation  in  this  context  are  not  well  developed.  Therefore  it  is  difficult  to  assess  the  impact  of 
the  current  set  of  COTS  products.  An  adjunct  velocity  tracker  may  provide  improved 
performance  should  that  prove  to  be  necessary. 
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Our  recommendation  is  that  the  FAA  develop  a  set  of  quantitative  performance  criteria  that  it 
desires  from  a  fusion  tracker.  If  reliability,  robustness,  and  smooth  transitions  among  sensors  are 
the  principal  criteria,  the  current  COTS  products  tested  are  suitable,  given  appropriate  tuning  and 
adaptation.  However,  if  the  desire  is  significant  improvement  in  aircraft  position  and  velocity 
estimation,  additional  development  may  be  required. 
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Figure  61 .  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  CD2  Data  -  COTS  B . 61 

Figure  62.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  B . 61 

Figure  63.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  B . 62 

Figure  64.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  B . 62 

Figure  65.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  CD2  Data  -  COTS  C . 63 

Figure  66.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  C . 63 

Figure  67.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  C . 64 

Figure  68.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  C . 64 

Figure  69.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  CD2  Data  -  COTS  D . 65 

Figure  70.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  D . 65 

Figure  71.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  D . 66 

Figure  72.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  D . 66 

Figure  73.  Computed  Turn  Rate  Values  -  Turn  200-230  Seconds  @  3  Deg/Sec  -  CD2  Data  -  COTS  D . 67 

Figure  74.  Computed  Turn  Rate  Values  -  Turn  200-230  Seconds  @  3  Deg/Sec  -  Mode  S  Data  -  COTS  D . 67 

Figure  75.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

CD2  Data -COTS  A . . 

Figure  76.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

CD2  Data  -  COTS  A . 68 

Figure  77.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  CD2 

Data  -  COTS  A . 68 

Figure  78.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Mode  S  Data  -  COTS  A . 69 

Figure  79.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  A . 69 

Figure  80.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S 

Data  -  COTS  A . 69 

Figure  81.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

CD2  Data  -  COTS  B . . 

Figure  82.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

CD2  Data  -  COTS  B . . 

Figure  83.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  CD2 

Data  -  COTS  B . . 

Figure  84.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Mode  S  Data  -  COTS  B . . 

Figure  85.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  B . . 

Figure  86.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S 

Data  -  COTS  B .  yi 
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Figure  88.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 
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Figure  89.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  CD2 

Data  -  COTS  C.  . . . 
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Figure  90.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Mode  S  Data -COTS  C . 73 

Figure  91 .  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -COTS  C. . 73 

Figure  92.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S 

Data- COTS  C . 73 

Figure  93.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

CD2  Data -COTS  D . 74 

Figure  94.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

CD2  Data -COTS  D . 74 

Figure  95.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  CD2 

Data -COTS  D . 74 

Figure  96.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Mode  S  Data  -  COTS  D . 75 

Figure  97.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  D . 75 

Figure  98.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S 

Data -COTS  D . 75 

Figure  99.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  CD2  Data  - 

COTS  A . 76 
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Figure  101.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  CD2  Data  - 

COTS  A . 76 

Figure  102.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  Mode  S 

Data -COTS  A . 77 

Figure  103.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  Mode  S 

Data -COTS  A . 77 

Figure  104.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  Mode  S  Data  - 

COTS  A . 77 

Figure  105.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  CD2  Data  - 

COTS  B . 78 

Figure  1 06.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  CD2  Data 

-COTS  B . 78 

Figure  107.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  CD2  Data  - 

COTS  B . 78 

Figure  108.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  Mode  S 

Data -COTS  B . 79 

Figure  109.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  Mode  S 

Data -COTS  B . 79 

Figure  1 10.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  Mode  S  Data  - 

COTS  B . 79 

Figure  111.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  CD2  Data  - 

COTS  C . 80 

Figure  112.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  CD2  Data 

-  COTS  . . 80 

Figure  113.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  CD2  Data  - 

COTS  . . 80 

Figure  1 14.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  Mode  S 

Data -COTS  C . 81 

Figure  115.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  Mode  S 

Data  -  COTS  . . 81 

Figure  1 16.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  Mode  S  Data  - 

COTS  . . 81 

Figure  1 17.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  CD2  Data  - 

COTS  . . 82 
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Figure  118.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  - 

CD2  Data  -  COTS  D . 82 

Figure  119.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  CD2  Data  - 

COTS  D . 82 

Figure  120.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle  Error  -  Mode  S 

Data  -  COTS  D . 83 

Figure  121.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed  Error  -  Mode  S 

Data  -  COTS  D . 83 

Figure  122.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  -  Mode  S  Data  - 

COTS  D . 83 

Figure  123.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  CD2  Data  -  COTS  A . 84 

Figure  124.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  CD2  Data  - 

COTS  A . 84 

Figure  125.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  CD2  Data  -  COTS  A . 84 

Figure  126.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  Mode  S  Data  - 

COTS  A . 85 

Figure  127.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  A . 85 

Figure  128.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S  Data  -  COTS  A . 85 

Figure  129.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  CD2  Data  -  COTS  B . 86 

Figure  130.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  CD2  Data  - 

COTS  B . 86 

Figure  131.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  CD2  Data  -  COTS  B . 86 

Figure  132.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  Mode  S  Data  - 

COTS  B . 87 

Figure  133.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  B . 87 

Figure  134.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S  Data  -  COTS  B . 87 

Figure  135.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  CD2  Data  -  COTS  C . 88 

Figure  136.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  CD2  Data  - 

COTS  C . 88 

Figure  137.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  CD2  Data  -  COTS  C . 88 

Figure  138.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  Mode  S  Data  - 
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Figure  139.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  C . 89 

Figure  140.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S  Data  -  COTS  C . 89 

Figure  141.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  CD2  Data  -  COTS  D . 90 

Figure  142.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  CD2  Data  - 

COTS  D . 90 

Figure  143.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  CD2  Data  -  COTS  D . 90 

Figure  144.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  -  Mode  S  Data  - 

COTS  D . 91 

Figure  145.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  D . 91 

Figure  146.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  -  Mode  S  Data  -  COTS  D . 91 

Figure  147.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  CD2  Data  -  COTS  A . 92 

Figure  148.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  A . 92 

Figure  149.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  CD2  Data  -  COTS  A . 92 

Figure  150.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  A . 93 

Figure  151.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  A . 93 

Figure  152.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  A . 93 

Figure  153.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  CD2  Data  -  COTS  B . 94 

Figure  154.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  B . 94 

Figure  155.  Outlier  Test  (Outlier  at  Time  100) -Position  Error -CD2  Data- COTS  B . 94 

Figure  156.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  B . 95 
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Figure  157.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  B . 95 

Figure  158.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  B . 95 
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Figure  161.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  CD2  Data  -  COTS  C . 96 

Figure  162.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  C . 97 
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Figure  192.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors  - 
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1.  FUSION  TRACKING  OVERVIEW 


1.1  INTRODUCTION 

Current  FAA  automation  systems  use  reports  from  a  single  beacon  sensor  to  determine  an 
aircraft’s  position,  altitude,  and  identification.  In  en-route  automation,  should  reports 
exist  from  overlapping  sensors,  one  sensor  is  chosen  as  the  preferred  sensor  and  the 
others  are  assigned  backup  supplementary  roles;  these  assignments  are  pre-defined  for 
each  sector  of  a  geographic  grid  termed  “radar  sort  boxes’^  Data  from  supplemental 
sensors  are  only  referenced  when  the  preferred  sensor  reports  are  missing.  This  system 
of  operation  is  named  “mosaicing.”  While  mosaicing  does  accomplish  integration  of 
surveillance  data  from  multiple  sensors,  it  has  significant  limitations.  For  example, 
differences  between  position  reports  for  the  same  aircraft  from  different  sensors  normally 
exist  due  to  sensor  errors,  including  biases.  These  errors  produce  significant  “hops”  or 
“stitching”  in  the  aircraft  positions  when  the  preferred  sensor  changes  or  when  the 
supplemental  sensor  is  used.  The  presence  of  these  phenomena  cause  controllers  to 
employ  increased  separation  standards  in  affected  regions,  significantly  reducing  the 
effective  airspace  capacity. 

In  recent  years,  with  the  advent  of  faster  computers,  new  techniques  for  multi-sensor 
surveillance  integration  have  been  developed.  While  the  techniques  vary  with  the 
individual  developer,  they  have  in  common  the  property  that  surveillance  reports  from 
multiple  sensors  are  combined  into  a  single  track  for  each  aircraft.  These  techmques  are 
collectively  referred  to  as  multi-sensor  “fusion”.  Like  mosaicing,  fusion  provides  an 
automatic  backup  in  the  event  of  a  single  sensor  failure  or  missing  data.  The  fact  that 
fusion  algorithms  make  use  of  all  sensor  data  to  form  a  single  track  is  often  cited  by 
fusion  developers  as  the  source  of  significant  surveillance  improvements  such  as  more 
rapid  track  initiation,  higher  update  rate,  and  more  accurate  position  and  velocity 
estimates.  An  additional  potential  improvement  is  the  elimination  of  discontinuities  at 
sensor  boundaries.  To  the  extent  that  separation  standards  are  defined  by  surveillance 
performance,  improvement  in  that  performance  will  increase  the  effective  capacity  of  the 
affected  airspace.  The  fundamental  technical  question  is;  what  quantitative  surveillance 
improvements  are  possible,  given  the  current  state  of  industry  skill  in  multi-sensor  fusion 
trackers? 

Lincoln  Laboratory  was  tasked  by  the  FAA  to  measure  the  performance  of  a 
representative  sample  of  current  commercial  off-the-shelf  (COTS)  fusion  trackers.  This 
effort  included  cataloging  the  companies  that  have  available  ATC  fusion  trackers, 
acquiring  executable  tracker  images  from  as  many  as  possible  of  these  companies, 
preparing  test  data  sets  (real  and  simulated)  to  input  to  these  trackers,  running  the 
commercial  tracker  code  on  the  test  sets,  and  evaluating  the  performance  achieved. 

This  report  presents  an  overall  review  of  the  state-of-the-art  of  fusion  trackers  as  applied 
to  the  FAA  surveillance  problem.  Average  statistics  of  performance,  as  well  as 
performance  in  special  situations,  are  included.  In  each  case,  the  performance  of  fusion 
is  compared  against  the  performance  of  single  sensor  and  mosaiced  tracking.  Thus  the 
advantages  and  disadvantages  of  fusion  will  be  evident.  The  statistics  may  also  permit 
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the  generation  of  a  fusion  tracker  specification  should  the  FAA  decide  to  procure  one  as 
part  of  a  future  automation  system. 

The  intent  of  this  study  is  to  provide  the  FAA  with  the  “state  of  the  shelf’  of  fusion 
trackers.  Therefore,  there  is  no  need  to  specifically  identify  a  particular  performance 
result  with  a  given  manufacturer  in  this  report.  In  exchange  for  their  participation, 
Lincoln  Laboratory  agreed  to  provide  each  manufacturer  with  specific  performance  data 
on  its  own  tracker  while  reporting  here  data  that  is  not  identified  by  manufacturer. 

There  were  two  major  classes  of  fusion  trackers  represented  in  the  industry  sample.  The 
first  formulates  tracks  from  each  individual  sensor  and  then  combines  the  tracks  into  a 
single  track  (track-to-track  fusion).  The  second  combines  the  reports  from  all  sensors 
directly  into  a  single  track  (report-to-track  fusion).  A  comparison  of  the  two  techniques, 
as  implemented  by  the  manufacturers,  is  provided  when  appropriate. 

All  of  the  COTS  trackers  were  Kalman  filters.  Although  a  Kalman  filter  is  the  theoretical 
optimum  tracker,  this  statement  is  true  only  if  the  aircraft  trajectories  are  modeled 
correctly  and  enough  states  exist  in  the  filter  to  match  the  model.  The  usual  linear 
Kalman  filter,  therefore,  will  have  problems  with  aircraft  maneuvers.  Also  least-mean- 
square  error,  the  measure  optimized  by  a  Kalman  filter,  is  not  necessarily  the  best 
indicator  of  performance.  For  a  simple  counter-example,  consider  the  tracker  options  of 
Figure  1.  Although  tracker  1  is  "optimum",  tracker  2  would  be  preferred  for  ATC 
applications,  where  maximum  error  is  more  important  for  determining  separation 
standards  than  is  average  error.  In  conclusion,  non-Kalman  filter  trackers  could  possibly 
be  superior  for  this  application,  even  though  none  were  offered. 

1.2  TRACKER  FUNCTIONS 

A  tracker  in  a  surveillance  and  automation  system  performs  several  functions,  of  which 
the  most  important  are: 

1.  Position  estimation 

2.  Velocity  (track  angle  and  ground  speed)  estimation 

3.  False  alarm  rejection. 

The  first  two  functions  depend  upon  the  ability  of  the  tracker  to  smooth  noisy  position 
measurements  made  by  the  surveillance  sensor,  while  the  third  fimction  depends  upon  the 
ability  of  the  tracker  to  correlate  reports  corresponding  to  the  same  aircraft  and 
distinguish  them  from  other,  nearby  reports.  A  tracker  can  have  as  inputs  either  single 
sensor  reports  or  reports  from  multiple  sensors;  the  latter  type  is  a  fusion  tracker. 

In  theory,  a  fusion  tracker  should  provide  improved  position  and  velocity  estimates 
because  it  has  more  reports  (measurements)  available  to  it.  The  higher  data  input  rate  can 
also  be  used  to  produce  a  higher  track  update  rate.  False  alarm  rejection  should  be 
improved  with  fusion  because  a  potential  false  aircraft  report  from  one  sensor  can  be 
checked  against  the  data  from  other  sensors  viewing  the  same  airspace  -  if  none  of  them 
see  the  aircraft,  the  report  can  be  dropped  with  greater  certainty. 
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Surveillance  attributes  that  complicate  the  job  of  a  fusion  tracker  include  aircraft  turns, 
false  alarm  reports,  garbled  report  information,  split  and  merged  reports,  and  system 
biases  and  registration  errors.  Although  registration  errors  can  be  reduced  by  careful 
calibration,  many  of  the  dozens  of  other  bias  sources  are  difficult  if  not  impossible  to 
compute  and  remove.  A  previous  Lincoln  project  [1]  attempted  to  use  recorded  data  to 
estimate  these  residual  biases,  but  the  interrelations  between  them  prevented  complete 
success.  More  recent  proprietary  algorithms  exist  for  removal  of  biases,  and  testing  of 
the  COTS  trackers  included  detecting  their  presence.  Because  of  the  existence  of  biases, 
a  poorly  designed  fusion  tracker  can  perform  worse  than  a  single-sensor  tracker  on  the 
same  input  data. 

Normally,  the  generation  of  an  aircraft  track  is  a  two-step  process.  First,  individual 
reports  must  be  associated  with  a  track  (report-to-track  association).  Then  the  reports  are 
assembled  into  a  track  (tracking).  As  discussed  below,  report-to-track  association  is  not 
always  perfect  and  can  introduce  errors  whose  impact  must  be  addressed  in  the  tracker 
implementation.  Clearly,  the  performance  of  a  tracker  is  dependent  upon  the  quality  of 
the  data  received  from  the  sensor.  At  present,  the  constraints  of  the  NAS  commumcation 
protocols  (CD2)  used  to  disseminate  surveillance  data  to  NAS  automation  cause 
significant  data  to  be  eliminated  -  both  significant  bits  in  the  measurements  (4  for 
terminal  sensor  range  and  5  for  enroute  sensor  range,  4  for  azimuth,  and  2  for  altitude), 
and  data  fields  that  would  greatly  aid  the  report-to-track  correlation  process.  Since  this 
loss  handicaps  the  tracker  in  the  NAS  automation  system,  it  is  important  that  the  impact 
of  these  performance  constraints  be  measured  as  part  of  this  evaluation.  Also,  due  to 
performance  coupling,  the  performance  of  the  report-to-track  association  process  and  the 
tracker  must  be  evaluated  in  combination.  The  focus  of  the  remainder  of  this  paper  is  on 
the  tracker  subsystem. 

It  is  important  to  note  that  the  tracking  done  in  the  NAS  automation  system  has  a 
relatively  long  prediction  time  horizon  (30-60  seconds)  compared  to  the  scan-to-scan 
processing  (often  also  called  tracking)  performed  in  the  sensor  itself  Sensor  tracking  is 
used  to  clean  up  uncertainty  in  the  data  and  to  significantly  reduce  the  false  alarms  sent  to 
automation.  The  automation  tracking  discussed  here  follows  the  sensor  tracking  but  is  in 
no  way  intended  to  replace  it. 

1.3  TRACKER  ANALYSIS  METHODOLOGY 

Analysis  of  a  fusion  tracker  should  measure  its  performance  in  several  categories,  of 
which  the  most  important  are: 

1 .  General  accuracy  -  position  and  velocity  in  straight  flight 

2.  How  quickly  it  responds  to  turns  -  and  accuracy  of  turn  data 

3.  How  long  it  takes  to  initiate  tracks  -  and  number  of  false  alarms 

4.  How  quickly  it  recovers  from  erroneous  correlations 

5.  Its  ability  to  handle  system  biases 

6.  Altitude  and  altitude  rate  estimation 
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The  analysis  should  be  made  on  both  recorded  and  simulated  inputs,  and  with  both  single 
sensor  and  multiple  sensor  databases  for  comparison.  The  recorded  data  runs  indicate 
qualitatively,  in  an  overall  average  sense,  how  well  the  trackers  perform  on  real  ATC 
data,  and  show  how  much  benefit  may  accrue  with  their  introduction.  The  simulation 
runs  permit  detailed  quantitative  measures  of  each  aspect  of  tracker  performance,  since 
ground  truth  is  known  and  specific  situations  can  be  generated. 

The  most  important  feature  of  a  tracker  is  its  ability  to  provide  track  angle  and  ground 
speed  information  on  an  aircraft.  It  does  this  by  smoothing  the  raw  measured  positions  of 
reports  in  a  filter  (usually  a  Kalman  filter).  Since  aircraft  tend  to  spend  most  of  their  time 
in  straight-line  flight,  the  key  filter  performance  characteristic  is  the  accuracy  of  position 
and  velocity  state  vectors  during  straight-line  flight. 

With  recorded  real  data,  scan-to-scan  changes  in  track  angle  and  ground  speed  can  be 
computed  to  measure  track  stability  as  well  as  time  to  settle  down  at  initiation.  Multi¬ 
sensor  fusion  tracking  will  have  more  reports  available  for  smoothing  (higher  data  rate). 
This  would  seem  to  be  an  advantage,  especially  for  altitude  tracking  during  climbs  and 
descents,  but  system  biases  can  actually  cause  fusion  tracking  to  degrade  below  mosaic 
tracking  if  not  accounted  for  by  adjustment  algorithms.  For  example,  the  mathematical 
foundation  for  a  Kalman  filter  ensures  optimal  performance  for  inputs  with  zero-mean 
Gaussian  noise.  If  the  true  noise  in  the  input  measurements  is  non-zero  mean  (contains 
biases)  and/or  non-Gaussian,  the  output  of  the  Kalman  filter  is  no  longer  optimal  and  may 
be  noisier  than  the  input  data.  Thus,  tracking  comparison  of  fusion  against  single  sensor 
data  was  included  in  the  test  plan. 

With  known  inputs  —  such  as  simulated  data  with  typical  sensor  measurement  noise  — 
performance  in  straight  flight  can  be  isolated  and  measured.  Thus,  the  average  and 
variance  of  position  and  velocity  errors  can  be  determined.  Both  types  of  tests  were 
performed. 

Although  turns  are  less  frequent,  they  cannot  be  ignored.  Safety  algorithms  such  as 
Conflict  Alert  depend  upon  knowing  when  a  turn  is  in  progress  to  detect  possible  future 
reductions  in  separation  below  acceptable  levels.  Thus,  the  ability  of  a  filter  to  note  the 
onset  and  end  of  a  turn  as  soon  as  possible  is  important.  In  particular,  its  time  to  recover 
stability  after  a  turn  is  an  important  tracker  characteristic.  In  the  worst  case,  loss  of 
positional  accuracy  in  a  turn  due  to  an  undershoot  of  the  ground  track  angle  estimate  can 
cause  the  tracker  to  drop  the  track,  removing  the  aircraft  from  the  automation  system 
(although  reports  will  continue  to  appear  on  the  display).  Finally,  accurate  calculation  of 
the  rate  of  turn  helps  prediction  algorithms  make  their  decisions.  To  adequately  perform 
these  functions,  the  filter  should  ideally  contain  turn  states,  such  as  occurs  in  IMM 
(Interacting  Multiple  Model)  Kalman  filters. 

Testing  turn  performance  requires  known  inputs,  so  that  delays  in  state  changes  and  turn 
rate  accuracy  can  be  measured.  Simulated  single  and  multiple  sensor  data  with  typical 
noise,  in  which  each  aircraft  flies  straight  for  200  seconds  to  establish  steady  state,  then 
performs  a  90-degree  turn  at  a  standard  3  deg/sec  turn  rate,  followed  by  return  to  straight 
flight,  were  used  to  measure  turn  performance.  Statistics  including  track  angle  error, 
ground  speed  error,  and  positional  error  versus  time  in  turn  were  generated.  Also,  time  to 
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detect  beginning  and  end  of  maneuvers  with  multi-sensor  data  was  compared  against 
single  sensor  performance. 

Track  initiation  with  multiple  sensor  tracking  can  be  more  complex  than  that  for  single 
sensor  tracking.  The  potential  for  false  alarm  tracks  is  also  increased  due  to  the  large 
number  of  “cross-sensor”  report  pairings.  A  properly  designed  fusion  tracker  should 
initiate  tracks  at  least  as  fast  as  the  single  sensor  case,  and  hopefully  faster,  with  no 
additional  false  alarm  tracks. 

The  test  for  this  performance  requirement  was  to  compare  the  single  and  multi-sensor 
results  for  time  to  initiate  tracks.  Both  recorded  and  simulated  data  were  used;  the 
recorded  data  indicated  “real-life”  performance  while  the  simulated  data  permitted  tests 
on  specially  constructed  situations.  Examples  of  such  situations  include  number  of 
sensors  and  percent  of  coast  scans.  No  false  alarm  performance  was  tested  in  this  study 
due  to  the  complexity  of  the  required  tests  and  analysis  routines,  especially  with  the 
number  of  trial  tracks  used  by  each  manufacturer  (which  mimic  false  tracks). 

No  tracker  can  attain  perfect  correlation  -  it  will  on  occasion  select  for  the  track  a  report 
that  does  not  correspond  to  the  actual  aircraft  position.  This  report  can  correspond  to 
another  nearby  aircraft,  it  can  be  a  false  alarm,  or  it  can  have  a  bad  position  due  to  errors 
in  the  report  generation  algorithm.  Such  a  report  has  the  potential  to  significantly  deviate 
the  track  position  or  velocity.  A  well-designed  tracker  should  have  the  ability  to  detect 
such  an  event  when  good  data  next  occurs,  and  recover  from  the  errors  caused  by  the  bad 
point.  A  particularly  desirable  implementation  is  to  check  each  suspect  report  on  the  next 
scan  when  hindsight  is  available;  if  it  is  verified  as  a  correlation  error,  the  tracker  should 
revert  to  its  previous  values  prior  to  the  error,  and  update  correctly  with  the  new  good 
report. 

A  method  of  testing  this  performance  characteristic  with  simulated  inputs  was  to  create  a 
set  of  reports  representing  straight-line  flight.  Then,  the  position  of  one  report  at  a  known 
time  was  significantly  altered,  and  the  resulting  data  fed  through  the  tracker.  The  errors 
in  the  predictions  recorded  after  the  bad  report,  compared  to  those  prior  to  the  outlier, 
indicate  the  tracker's  resilience  to  correlation  errors. 

System  biases  will  always  exist  in  a  multi-sensor  fusion  system.  The  fusion  tracker  must 
be  able  to  deal  with  them,  either  by  containing  bias-resistant  algorithms  or  by  computing 
in  real  time  the  biases  and  then  subtracting  them  from  the  input  reports.  Since  bias- 
resistant  algorithms  are  more  difficult  to  design,  and  often  have  inferior  performance, 
most  contractors  that  deal  -with  biases  do  so  by  computing  them. 

A  method  of  testing  this  performance  characteristic  was  to  play  a  biased  data  scenario 
through  the  trackers  twice;  once  at  the  beginning  of  the  run,  and  then  again  after  an  hour 
of  general  aircraft  data.  If  the  output  data  and  predictions  are  more  accurate  the  second 
time  than  the  first  time,  bias  calculation  and  adjustment  algorithms  were  employed  by  the 
tracker.  Of  course,  one  hour  of  delay  is  generally  not  enough  time  for  the  tracker  to 
complete  its  bias  adjustments,  so  the  final  accuracies  that  would  be  achieved  are  not 
known.  The  comparison  of  unbiased  to  biased  results  indicate  how  important  careful 
location  and  calibration  of  FAA  sensors  is  to  a  fusion  tracker 
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Bias  performance  was  also  tested  to  see  if  the  track  "hops"  discussed  earlier  that  are 
encountered  with  mosaic  systems  were  reduced  with  fusion.  The  method  of  testing 
employed  was  to  input  straight  flight  biased  data  into  each  tracker,  such  that  the  preferred 
sensor  reports  for  each  aircraft  were  curtailed  at  a  known  time.  Then  comparing  the 
results  of  fusion  tracking  to  that  of  mosaiced  tracking  indicated  the  improvement  attained 
with  fusion. 

Altitude  and  altitude  rate  estimation  are  important  requirements  of  a  tracker,  as  safety 
functions  such  as  Conflict  Alert  and  MSAW  depend  upon  these  quantities.  This  report 
did  not  study  altitude  tracking,  as  the  altitude  algorithms  are  usually  separate  from  the  x,y 
Kalman  filter  and  are  not  significantly  impacted  by  fusion  versus  mosaic  considerations. 
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2.  FUSION  TRACKER  ACQUISITION 


Lincoln  was  able  to  contact  seven  companies  that  have  fusion  trackers  suitable  for  ATC 
surveillance  applications.  These  companies,  their  available  trackers,  and  the  result  of 
Lincoln’s  request  to  obtain  a  test  version  are  as  follows: 

1.  Lockheed-Martin  (Minnesota)  -  MicroEARTS  tracker  -  executable  image 
running  at  Lincoln 

2.  Telephonies  -  ATC  tracker  used  abroad  -  executable  image  running  at  Lincoln 

3.  Etirocontrol  -  operational  ATC  fusion  tracker  (ARTAS)  in  Europe  -  executable 
image  running  at  Lincoln 

4.  Raytheon  -  STARS  tracker  -  executable  image  running  at  Lincoln 

5.  Sensis  -  ATC  tracker  -  company  declined  request  to  supply  tracker 

6.  Lockheed-Martin  (Owego)  -  best-of-breed  AW  ACS  tracker  -  no  tracker 
supplied 

7.  Alenia  -  operational  ATC  fusion  tracker  in  Italy  -  no  tracker  supplied 

The  fusion  evaluation  effort  described  in  this  report  has  made  use  of  the  first  four  COTS 
trackers.  As  per  Lincoln's  agreement  with  the  companies,  no  name  appears  on  any  of  the 
result  figures.  Instead,  the  letters  A,B,C,D  were  employed.  To  prevent  inferences  on  a 
letter-to-company  match,  the  order  of  the  companies  was  varied  from  section  to  section. 

2.1  COTS  TRACKER  PREPARATION 

As  expected,  all  of  the  COTS  trackers  required  extensive  preparation  by  the 
manufacturers  before  they  could  be  applied  to  our  ATC  databases.  The  first  step,  which 
we  expected,  was  that  each  tracker  had  to  be  told  of  the  characteristics  of  each  sensor  in 
our  ensemble:  location,  scan  rate,  and  data  accuracy.  Further  steps,  some  of  which  came 
as  a  surprise,  included  modification  of  tracker  parameters  to  be  able  to  handle  the 
scenarios,  real  and  simulated,  that  we  used  for  the  tests.  Also,  we  had  to  modify  our  data 
inputs  to  match  the  peculiarities  of  some  of  the  trackers,  and  matching  stereographic 
projections  was  a  challenge  for  trackers  that  did  not  use  lat/long  coordinates. 

In  summary,  none  of  the  trackers  could  be  used  without  some  modifications  by  the 
companies  or  us.  One  of  the  trackers  required  several  new  builds  to  operate  properly, 
while  two  others  required  numerous  back  and  forth  dialogs  between  Lincoln  and  the 
companies  to  reach  an  operational  state  after  modifications  were  implemented. 

2.2  COTS  TRACKER  TYPES 

All  the  COTS  trackers  employed  a  Kalman  filter  implementation,  although  the  details 
varied  from  company  to  company.  One  company  chose  a  track-to-track  system,  in  which 
each  sensor’s  data  is  tracked  separately  in  its  own  Kalman  filter.  These  filter  outputs  are 
then  weighted  averaged  to  produce  the  final  output  provided  to  the  user.  The  other  three 
companies  selected  report-to-track  fusion,  in  which  all  reports  from  all  sensors  are  fed  to 
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the  same  Kalman  filter.  There  was  however  a  divergence  in  the  filter  states  defined  in 
these  filters,  in  that  one  of  the  companies  introduced  turn  rate  as  a  state  while  the  other 
two  adopted  a  linear  filter. 

The  filter  differences  also  affected  the  types  of  data  runs  we  could  attempt  in  two  ways. 
First,  the  track-to-track  filter  could  not  be  fed  mosaic  data,  as  data  from  a  new  sensor 
requires  many  scans  to  re-initialize  the  output  stream.  Second,  one  of  the  other  trackers 
had  input  format  restrictions  that  did  not  permit  the  definition  of  a  long-range  terminal 
sensor,  which  barred  the  use  of  SURVSEF  (the  production  Mode  S  sensor  operated  at 
Lincoln  Laboratory)  as  a  testable  sensor. 

The  position  and  velocity  information  contained  in  the  tracker  output  reports  were 
analyzed  in  this  study.  Some  of  the  COTS  products  have  additional  fields  internal  to  the 
tracker  that  may  influence  the  information  passed  to  automation  safety  functions. 
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3.  RECORDED  DATA  BASE  GENERATION 


Last  year,  Lincoln  obtained,  with  the  assistance  of  FAA  personnel,  a  collection  of  report 
data  from  many  of  the  beacon  sensors  in  the  New  England  region,  including  the  Mode  S 
sensor  operated  by  Lincoln  (operated  at  a  250-mn  range),  terminal  sensors  at  regional 
airports,  and  enroute  sensors  reporting  to  the  New  England  center.  Figure  2  presents  the 
location  of  the  sensors  employed;  their  specifications  are  listed  below  as  part  of  the 
complete  list  in  the  simulation  section.  Figure  3  plots  8  sample  minutes  of  the  recorded 
data  reports.  These  data  have  been  reduced  and  converted  to  a  form  that  can  be  input  to 
the  COTS  algorithms. 

Since  the  non-Mode  S  sensors  had  no  time  included  in  the  reports,  Lincoln  was  required 
to  develop  a  time-alignment  program  to  assign  a  common  time  to  all  reports.  This  was 
accomplished  by  including  a  Lincoln  aircraft  in  the  database,  which  performed  several 
high  rate  climbs  and  descents.  By  slotting  reports  based  on  reported  altitude  -  for 
example,  a  sensor  7  report  at  12,500  feet  must  have  occurred  between  sensor  1  reports  at 
12,400  feet  and  12,600  feet  -  and  by  iterating  over  subsets  of  sensors  to  find  a  global 
solution,  the  time  offset  of  each  sensor  relative  to  sensor  1  was  determined.  Then  using 
the  offset,  and  assuming  a  constant  antenna  rotation  rate  to  convert  azimuth  to  time-in- 
scan,  time  was  added  to  all  reports. 

Initial  examination  of  the  recorded  data  indicated  the  presence  of  significant  inter-sensor 
registration  errors.  For  example.  Figure  4  plots  a  sample  segment  of  data  from  3  sensors, 
in  which  positional  offsets  are  obvious.  Simple  analysis  revealed  that  azimuth  bias 
(incorrect  northmark  calibration)  was  the  overriding  culprit  producing  these  errors. 
Correcting  the  azimuth  biases  of  each  sensor  by  aligning  the  data  in  the  best  mean-square 
sense  removed  most  of  the  errors;  Figure  5  illustrates  the  new  plot.  Since  we  assume  that 
accurate  azimuth  calibration  will  be  made  by  the  FAA  for  all  sensors  before  fusion 
trackers  are  operated,  the  recorded  data  points  were  adjusted  by  the  amount  of  the 
computed  azimuth  biases  before  the  fusion  tests  were  performed. 

Residual  biases  still  existed  in  the  sensor  data.  An  attempt  was  made  by  Lincoln  to 
estimate  these  biases  for  a  subset  of  the  sensors  assuming  that  azimuth  bias  and  sensor 
location  were  the  main  contributing  factors.  The  results  are  presented  in  Figure  6.  These 
numbers  are  not  "correct"  since,  as  explained  above,  attempts  to  compute  actual  biases 
have  not  succeeded  to  date,  but  they  provide  an  indication  of  the  magnitude  of  the  biases 
existing  in  the  recorded  data.  GPS  truth  was  used  to  help  in  the  bias  estimation  effort. 
Unfortunately,  GPS  was  only  available  for  a  single  aircraft;  GPS  data  from  several 
aircraft  located  around  the  surveillance  region  would  provide  more  reliable  results. 

3.1  TRACKER  SMOOTHING  OF  RECORDED  DATA 

The  recorded  data  set  was  used  to  generate  statistics  on  typical  performance  for  “real” 
scenarios.  Since  truth  is  known  for  only  one  aircraft  (the  GPS  aircraft,  analyzed  below), 
scan-to-scan  variations  of  track  angle  and  groimd  speed  replaced  average  errors  as  a 
measure  of  the  tracker's  smoothing  capabilities. 
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Multiple  sensor  inputs  provide  an  increased  number  of  reports  available  to  the  tracker. 
Although  this  is  assumed  to  be  a  positive  factor,  it  can  challenge  a  Kalman  filter  type  of 
tracker.  In  particular,  the  variable  update  rate  can  introduce  apparent  velocity  variations. 
The  closer  together  in  time  are  successive  reports,  the  greater  will  be  the  effect  of 
measurement  noise  and  biases. 

Figures  7  and  8  present  for  CD2  enroute  data  and  Mode  S  terminal  data  respectively  the 
noisiness  of  the  velocity  vectors  of  successive  report  pairs  as  a  function  of  the  smaller  of 
the  two  time  intervals.  (That  is,  3  successive  reports  are  joined  in  a  “connect  the  dots” 
manner,  and  the  two  vectors  are  compared).  It  is  clear  that  the  smaller  the  interval,  the 
greater  the  variation.  Note  in  addition  the  significantly  smaller  values  for  a  full  sensor 
scan  (12  seconds  for  enroute,  5  seconds  for  terminal),  in  which  the  successive  reports 
have  come  from  the  same  sensor.  This  illustrates  that  a  single  sensor  data  stream  is 
smoother  because  of  the  absence  of  inter-sensor  biases.  Finally  note  the  significant 
improvement  of  the  data  with  Mode  S  accuracy  over  that  of  CD2. 

It  is  these  inter-sensor  biases  that  cause  the  “hops”  in  the  current  enroute  mosaic  system. 
To  investigate  the  velocity  hops  that  would  occur  with  the  recorded  data  under  a  mosaic 
regimen,  the  preferred  reports  for  each  track  were  culled  from  the  data  base.  These 
reports  were  then  fed  to  two  of  the  three  report-to-track  COTS  trackers  (the  third  did  not 
supply  sensor  identification  of  output  reports).  Finally,  the  times  when  a  preferred  sensor 
swap  occurred  were  identified  and  analyzed. 

Figure  9  presents  the  performance  for  each  company  of  the  track  angle  and  ground  speed 
estimates  at  these  swap  times  as  compared  to  the  input  data.  The  input  data  numbers 
were  generated  as  above,  while  the  tracker  numbers  were  computed  as  the  differences  in 
tracker  values  before  and  after  the  swap.  As  seen,  both  trackers  were  capable  of 
significantly  smoothing  the  output  data,  nearly  eliminating  the  hop  problem  in  the 
velocity  sphere.  Position  hops  cannot  be  removed  when  transitioning  from  one  sensor 
coordinate  system  to  another,  but  these  are  much  less  bothersome  to  an  automation 
system  than  are  the  velocity  hops. 

Next  we  studied  the  smoothing  properties  of  the  trackers  in  the  general  situation.  Tests 
on  the  ability  of  the  COTS  trackers  to  smooth  the  recorded  reports  were  made  on  various 
subsets  of  the  sensor  data  to  study  the  effects  of  number  of  sensors,  sensor  update  rate, 
and  data  quality.  Four  cases  were  examined: 

1.  mosaicing  of  enroute  CD2  sensors  assuming  the  closest  sensor  to  be  the 
preferred, 

2.  fusion  tracking  of  all  enroute  CD2  sensor  data 

3.  tracking  of  single  terminal  sensor  Mode  S  data  (SURVSEF),  and 

4.  fusion  tracking  of  all  terminal  Mode  S  sensor  data. 

Figures  10  and  11  present  the  average  scan-to-scan  variation  of  the  tracker’s  output 
velocity  estimates  for  each  case,  the  statistics  selected  to  judge  the  smoothing  properties 
of  the  COTS  trackers. 
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It  should  be  noted  that  none  of  the  trackers  had  sufficient  time  in  this  test  to  fully  execute 
any  bias  adjustment  algorithms  they  may  contain.  Thus  the  performance  may  be  better  in 
an  operational  system  that  has  had  time  to  settle  down.  Bias  adjustment  performance  is 
addressed  in  detail  in  the  simulation  section  of  this  report. 

3.2  TRACKER  ACCURACY  ON  RECORDED  DATA 

As  discussed  above,  one  of  the  aircraft  in  the  recorded  data  set  had  a  GPS  receiver 
onboard  that  permitted  determination  of  its  true  position  and  velocity  at  each  second. 
Analyzing  the  tracker  outputs  on  this  aircraft,  velocity  and  position  errors  could  be 
computed.  All  4  sensor  cases  defined  above  were  used  to  test  the  COTS  tracker 
performances,  although  some  cases  could  not  be  run  for  some  of  the  companies  (mosaic 
is  not  possible  for  the  track-to-track  Kalman  filter,  and  SURVSEF  could  not  be  formatted 
to  meet  the  requirements  of  another  company).  Unfortunately,  the  aircraft  flew  at  a  low 
altitude,  so  multiple  sensor  coverage  was  minimal.  Thus  the  CD2  mosaic  and  CD2 
enroute  fused  cases  produced  nearly  similar  results,  as  did  the  SURVSEF  and  Mode  S 
terminal  fused  cases.  Differences  between  single  sensor  and  sensor  fusion  tracking 
performances  are  addressed  in  greater  detail  in  the  simulation  section. 

The  section  of  the  aircraft  trajectory  selected  for  analysis  consisted  of  2  turns  separating  3 
straight  sections.  Thus  performance  in  a  turn  as  well  as  recovery  after  its  completion 
could  be  observed.  The  results  are  presented  in  Figures  12  through  23,  where  track  angle, 
ground  speed,  and  position  error  versus  time  are  shown  for  each  company. 

COTS  A,  which  could  only  be  run  on  CD2  data,  exhibited  undershoot  in  track  angle 
during  turn  initiations,  followed  by  oscillatory  recovery  after  its  completion.  In  parallel, 
the  ground  speed  estimates  varied  widely  during  this  period,  generally  on  the  high  side, 
although  the  aircraft  was  flying  at  a  constant  speed.  Position  errors  also  varied,  although 
over  a  fairly  small  range. 

COTS  B  exhibited  similar  behavior  for  the  CD2  cases  as  COTS  A,  except  that  the 
positional  variations  were  significantly  greater  (the  0.4  nm  average  error  was  due  to 
differences  in  stereographic  projection  planes,  and  is  not  operationally  relevant).  The 
Mode  S  performance  however  was  exceedingly  accurate,  with  only  minor  track  angle 
imdershoot  and  virtually  no  ground  speed  or  positional  problems. 

COTS  C  had  major  problems  with  CD2  quality  data,  indicating  much  tuning  would  be 
required  before  its  operational  use.  The  track  angle  correction  during  turns  was  so  slow 
that  the  positional  error  grew  to  be  as  large  as  2  miles  (see  the  screen  image  below  as 
well).  The  Mode  S  tuning  was  significantly  better,  and  no  major  problems  were 
encountered  during  the  turns. 

Finally,  COTS  D  showed  track  angle  imdershoot  in  turns,  followed  by  slow  recovery, 
with  CD2  errors  more  pronounced  them  Mode  S.  Its  ground  speed  performance  had  no 
problems  in  the  first  turn,  but  the  incomplete  recovery  of  the  track  angle  before  the 
second  turn  initiation  caused  ground  speed  problems  in  that  turn.  The  positional  accuracy 
was  acceptable  in  all  cases. 
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To  provide  visual  images  of  the  performance  just  noted,  FUS-RAP  was  used  to  peruse 
the  results  of  tracking  the  recorded  data.  This  is  a  graphical  display  program  based  on  the 
9PAC  program  PAC-RAP  [2].  It  permits  placing  input  data,  truth  data  (if  present),  and 
tracker  output  data  together  on  the  screen.  By  noting  the  tracker  predicted  velocity 
vectors,  compared  to  the  observed  trajectories,  performance  can  be  inferred  for  ground 
speed  and  track  angle  errors. 

Figures  24  through  27  illustrate  for  this  maneuvering  aircraft  actual  screen  images  of  the 
tracker  predictions  for  CD2  data  processed  by  the  various  company  fusion  systems 
(COTS  D  outputs  more  frequently  than  once  per  scan).  The  undershoot  and  recovery 
properties  of  each  tracker,  noted  in  the  last  set  of  figures,  are  apparent  during  and  after 
the  turns.  Statistics  on  velocity  performance  in  turns  is  addressed  in  detail  below  for 
simulated  trajectories,  where  truth  is  known  for  an  ensemble  of  aircraft. 
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4.  SIMULATION  CAPABILITY 


Lincoln  Laboratory  has  developed  a  simulation  capability  that  was  used  to  generate  a 
variety  of  realistic  simulation  data  sets.  These  sets  were  used  to  test  performance  versus 
truth  (truth  in  real  data  being  unknown).  For  example,  performance  versus  system  biases 
was  examined  via  the  simulation.  Also,  special  simulation  data  sets  were  prepared  for 
tests  of  specialized  statistics  such  as  track  angle  error  in  turns.  In  this  project,  numerous 
analysis  runs  using  simulated  reports  have  been  carried  out. 

4.1  SIMULATION  SENSOR  BASE 

The  Lincoln  simulation  generates  data  from  all  enroute  sensors  currently  in  the  New 
England  region,  plus  all  terminal  sensors  at  New  England  airports,  plus  the  Lincoln 
production  Mode  S  sensor  that  existed  at  SURVSEF.  For  each  sensor,  both  a  non- 
Mode  S  CD2  format  version  and  a  Mode  S  advanced  format  version  are  defined,  so  that 
tests  dependent  upon  report  data  accuracy  can  be  performed.  The  complete  list  of 
sensors,  shown  geographically  by  Figure  28,  is  as  follows,  where  numbers  1  through  25 
represent  the  actual  current  sensor  type  and  assigned  number,  and  the  numbers  26 
through  50  represent  the  alternate  sensor  version: 
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ID 


Lat  deg. 


Lon  deg. 


Sensor 


Format 


Type 


Name 


LXM 

MAM 

NTC 

SAC 

PRM 

WLM 

SAM 

EEC 

DAC 

cue 

RIC 

RC2 

UTC 

STC 

BHC 

SKC 

CAC 


+42.45661 

+42.95001 

+42,03417 

+44.78171 

+41.72056 

+41.93847 

+44.78171 

+41.35694 

+42.63917 

+42.47472 

+40.87833 

+40.87833 

+43.34556 

+41.49036 

+44.62963 

+44.86139 

+46.88600 


-71.26644 

-71,29922 

-70.05417 

-73.06566 

-71.59861 

-72.68296 

-73.06566 

-76.29333 

-77.65944 

-72.96805 

-72.68722 

-72.68722 

-75.25000 

-74.10597 

-67.39549 

-69.51111 

-67.97164 


1 

2 

4 

5 

7 

8 

14 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 


Mode  S 
Mode  S 
CD2 
CD2 

Mode  S 
Mode  S 
Mode  S 
CD2 
CD2 
CD2 
CD2 
CD2 
CD2 
CD2 
CD2 
CD2 
CD2 


terminal 

terminal 

enroute 

enroute 

terminal 

terminal 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 


Lexington,  MA 
Manchester,  NH 
North  Truro,  MA 
St.  Albans,  VT 
Providence,  RI 
Windsor  Locks,  CT 
St.  Albans,  VT 
Benton.  PA 
Dansville.  NY 
Curamington,  MA 
Riverhead,  NY 
Riverhead,  NY 
Utica,  NY 
Stewart,  NY 
Bucks  Harbor ,  ME 
Skowhwgan ,  ME 
Caribou,  ME 


LXM 

MAM 

NTC 

SAC 

PRM 

WLM 

SAM 

BEC 

DAC 

cue 

RIC 

RC2 

UTC 

STC 

BHC 

SKC 


+42.45661 

+42.95001 

+42.03417 

+44.78171 

+41.72056 

+41.93847 

+44.78171 

+41.35694 

+42.63917 

+42.47472 

+40.87833 

+40.87833 

+43.34556 

+41.49036 

+44.62963 

+44.86139 

+46.88600 


-71.26644 

-71.29922 

-70.05417 

-73.06566 

-71.59861 

-72.68296 

-73.06566 

-76.29333 

-77.65944 

-72.96805 

-72.68722 

-72.68722 

-75.25000 

-74.10597 

-67.39549 

-69.51111 

-67.97164 


26 

27 

29 

30 

32 

33 
39 

41 

42 

43 

44 

45 

46 

47 


CD2 

CD2 

Mode  S 
Mode  S 
CD2 
CD2 
CD2 

Mode  S 
Mode  S 
Mode  S 
Mode  S 
Mode  S 
Mode  S 
Mode  S 


terminal 

terminal 

enroute 

enroute 

terminal 

terminal 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 

enroute 


Lexington,  MA 
Manchester,  NH 
North  Truro,  MA 
St.  Albans,  VT 
Providence,  RI 
Windsor  Locks,  CT 
St .  Albans ,  VT 
Benton.  PA 
Dansville.  NY 
Cummington.  MA 
Riverhead,  NY 
Riverhead,  NY 
Utica,  NY 
Stewart ,  NY 
Bucks  Harbor,  ME 
Skowhwgan,  ME 
Caribou,  ME 


CAC 


48 

49 

50 


Mode  S 
Mode  S 
Mode  S 


enroute 


The  assumed  data  characteristics  for  each  sensor  type  are  as  follows: 


type  of 
sensor 

range 

1 -sigma 

range 

quantum 

azimuth 

1-sigma 

azimuth 

quantum 

enroute  CD2 

100' 

1/8  nm 

3  mrad 

1  ACP 

terminal  CD2 

100' 

1/16  nm 

3  mrad 

1  ACP 

25' 

1/192  nm 

1  mrad 

1/16  ACP 

terminal  Mode  S 

25' 

1/192  nm 

1  mrad 

1/16  ACP 

Terminal  sensors  have  a  coverage  range  of  60  nautical  miles,  while  SURVSEF  and 
enroute  sensors  go  out  to  250  nautical  miles. 

4.2  SIMULATION  DATA  SETS 

The  basic  simulation  data  set  (denoted  as  BasicSim)  has  200  aircraft,  each  starting  with 
200  seconds  of  straight  constant  speed  flight,  followed  by  30  seconds  of  standard-rate 
turn  3°/sec  turning  right  flight,  followed  by  another  100  seconds  of  straight  flight.  All 
reports  have  standard  sigma  random  noise  depending  upon  sensor  type.  The  initial 
groimd  speeds  are  randomly  distributed  between  60  and  600  knots,  and  the  initial  track 
angles  are  randomly  distributed  between  0  and  360  degrees,  with  the  starting  positions 
randomly  selected  within  the  sensor  constellation.  Figure  29  depicts  these  200 
trajectories. 

The  simulation  generates  reports  for  all  of  the  sensors  defined  above.  A  culling  program 
selects  from  this  set  of  reports  those  corresponding  to  the  set  of  sensors  desired  for  the 
specific  test  (single  sensor,  mosaic,  all  CD2  enroute,  etc.).  When  a  mosaic  test  is  desired, 
the  culling  program  in  addition  determines  the  preferred  (closest)  and  supplemental  (next 
closest)  sensors  for  each  track  at  each  scan;  if  the  former  sensor  report  is  present,  it  is 
output,  else  the  latter  sensor  report  is  used. 

This  database  has  been  used  for  absolute  and  comparison  performance  tests  for  straight 
and  turning  flight  and  the  transitions  between  them.  The  results  achieved  are  presented 
later  in  this  paper. 

The  second  simulated  data  set,  of  one-hour  duration,  consists  of  random  trajectories,  with 
randomly  occurring  maneuvers  (turns,  accelerations,  climbs,  etc.  all  with  standard  airliner 
values),  and  a  time-varying  set  of  tracks.  Data  culling  is  performed  as  before.  This  set  is 
labeled  RandSim  for  random  simulation,  and  has  been  used  for  the  more  comprehensive 
fusion  tests  detailed  later. 

Specialized  simulation  sets  were  generated  as  needed  for  specific  tests;  these  are 
described  in  the  test  description  sections.  An  example  set  is  the  one  used  to  determine 
the  effect  of  and  recovery  from  bad  data  points.  This  involves  introducing  10-sigma  data 
error  points  at  known  times  for  each  track. 
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5.  ANALYSIS  SOFTWARE 


To  permit  the  operation  of  the  analysis  effort,  Lincoln  has  developed  and  coded  a 
package  of  analysis  software.  Both  graphical  and  statistical  programs  were  produced. 

The  graphical  tool  permits  observation  of  the  performance  of  the  fusion  tracker.  Input 
reports,  truth  (when  simulation  is  employed),  and  tracker  output  estimates  are  all 
displayable.  Thus  the  overall  performance  of  the  tracker  is  visible,  as  well  as  any  and  all 
instances  of  tracker  problems.  In  particular,  the  display  reveals  performance  areas  where 
further  testing  and  analysis  are  required,  while  the  ability  to  locate  problem  cases  helps 
direct  the  detailed  tracker  debugging  effort. 

The  analysis  programs  are  intended  to  generate '  statistical  measures  of  tracker 
performance,  both  overall  and  in  special  situations.  For  example,  the  average  track  angle 
error  of  the  tracker,  as  well  as  the  error  during  turns,  are  both  available. 

5.1  DATA  MERGING 

Because  each  COTS  tracker  has  a  different  output  format,  every  analysis  program  had  to 
be  preceded  by  a  re-formatting  program.  A  set  of  these  programs  was  created,  one  for 
each  company.  The  most  complex  of  these  was  the  Merge  routine,  which  was  required  to 
match  every  tracker  output  report  with  a  corresponding  simulation  input  report.  Time 
variations  and  differences  in  available  data  fields  complicated  these  programs. 

Examples  of  tracker-specific  problems  Merge  had  to  overcome  were  different  time 
references  and  coordinate  systems  from  the  simulation,  reports  output  at  different  times 
from  the  input  reports,  and  plots  and  tracks  output  in  different  files.  Thus,  some  Merges 
worked  with  3  files  instead  of  2,  and  one  had  to  interpolate  the  simulation  input  reports  to 
the  time  of  the  tracker  output.  Also,  one  company  had  its  ouQiut  times  expanded  by  2.5% 
relative  to  the  simulation  report  times  (that  is,  if  2  simulation  reports  differed  by 
100  seconds,  the  corresponding  tracker  output  times  differed  by  102.5  seconds).  As  a 
result,  every  Merge  program  was  quite  different  from  every  other  one. 

The  output  data  from  each  fusion  run  had  first  to  be  prepared  for  the  analysis  programs. 
The  preparation  depended  upon  the  type  of  output  provided  by  each  company;  some 
output  the  raw  input  reports  separately  from  the  smoothed  output  tracked  reports,  while 
others  presented  combined  ouqiut  reports.  In  the  former  case,  preparation  consisted  of 
separating  output  reports  and  output  tracks  into  separate  lists,  then  merging  the  reports 
and  tracks  into  pairings  (the  output  tracked  report  with  the  input  report  that  was  used  in 
its  generation),  and  finally  merging  each  pair  with  the  simulated  input  report  that  was 
used  to  create  it.  In  the  latter  case,  the  merge  was  performed  directly.  A  separate  merge 
program  was  required  for  each  company  due  to  the  lack  of  uniformity  in  the  fields 
provided  in  the  output  tracked  reports. 

For  each  company,  the  final  output  of  the  merge  was  a  file  of  all  merged  entities  in  a 
common  fusion  format  recognized  by  the  analysis  programs.  The  positions  in  this  format 
are  either  in  stereographic  x-y  or  in  lat-long  depending  upon  the  company  choice  of 
coordinates.  A  sample  section  of  a  merge  file  is  shovra  in  Figure  30.  Note  that  in  this 
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case,  as  occurred  for  all  COTS  trackers,  an  output  report  is  not  produced  for  each 
simulation  input. 

5.2  SIMULATION  ANALYSIS 

The  major  statistical  analysis  program  for  the  simulation  runs  computes  the  positional, 
ground  speed,  and  track  angle  error  for  each  merged  entity.  These  errors  were  then 
averaged  for  each  combination  of  flight  maneuver,  aircraft  ground  speed,  range  from 
closest  sensor,  and  most  importantly  time  into  the  maneuver.  This  program  permits 
graphs  of  such  statistics  as  time  to  settle  down  upon  track  initiation,  track  angle  error 
versus  time  in  turn,  ground  speed  error  versus  time  in  acceleration,  and  time  to  recover 
when  transitioning  from  one  type  of  maneuver  to  another. 
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6.  SIMULATION  RESULTS 


The  simulation  capability  was  employed  to  test  a  variety  of  COTS  tracker  characteristics. 
For  each,  a  tailored  data  set  was  generated,  whose  reports  produced  the  effect  to  be 
analyzed.  The  characteristics  tested  included  handling  of  turns,  resistance  to  biases, 
resilience  in  the  face  of  sensor  loss  or  bad  input  reports,  and  time  to  initiate  tracking  and 
adapt  to  and  recover  from  maneuvers. 

Although  the  simulation  data  sets  included  aircraft  whose  groimd  speeds  were  in  the 
interval  60-600  knots,  the  analysis  was  restricted  to  those  aircraft  having  a  minimum 
ground  speed  of  150  knots.  This  requirement  was  adopted  after  it  became  apparent  that 
the  trackers  had  significant  problems  with  the  slower  aircraft,  as  at  such  speeds  the  data 
noise  was  comparable  to  the  inter-report  motion.  Figures  31  and  32  illustrate  the  problem 
by  plotting  the  track  angle  error  for  straight  flight  as  a  function  of  time-in-track  for  both 
CD2  and  Mode  S  quality  data  for  one  of  the  COTS  trackers.  It  is  seen  that  the  trackers  on 
slow  speed  aircraft  failed  to  settle  down  even  after  200  seconds,  as  compared  to  settling 
times  of  less  than  60  seconds  for  faster  moving  aircraft.  Fortunately,  most  aircraft  in 
enroute  airspace  exceed  150  knots,  so  the  results  presented  below  for  the  trackers  are  in 
fact  representative  of  their  ability  to  handle  real  situations. 

6.1  BASICSIM  ENSEMBLE  RESULTS 

The  BasicSim  data  set  described  above  was  rim  through  each  commercial  fusion  tracker 
for  the  following  set  of  sensor  cases: 

1.  SURVSEF  only  (4.8  second  scan) 

2.  mosaicing  using  all  enroute  sensors  (each  12.0  second  scan) 

3.  fusion  using  all  enroute  sensors 

Two  exceptions  occurred  for  the  cases  run.  First,  the  COTS  C  tracker  as  provided  to  us 
was  incapable  of  running  SURVSEF  data,  as  long-range  terminal  sensor  was  not  an  input 
specification  supported  in  the  build.  Second,  the  track-to-track  filter  of  COTS  B  could 
not  be  run  in  a  mosaic  mode,  as  such  a  filter  will  terminate  tracking  for  a  full  initiation 
time  when  the  sensor  supplying  the  reports  is  changed.  Each  remaining  case  was  run  for 
both  CD2  and  Mode  S  quality  reports  as  subcases.  The  input  data  in  each  case  was 
imbiased;  the  effects  of  data  biases  are  addressed  in  the  next  section. 

For  each  case  run,  the  analysis  program  was  used  to  generate  the  absolute  track  angle, 
ground  speed,  and  position  errors  for  each  tracked  output  report.  These  errors  were 
averaged  over  all  tracks  for  each  5-second  interval  of  the  simulation.  The  results, 
presented  in  Figures  33  through  56,  indicate  how  well  each  tracker  performed  in  each 
situation.  In  particular,  because  of  the  nature  of  the  BasicSim  data  set,  the  following 
characteristics  are  evident  for  each  company  in  each  case: 

1 .  The  minimum  time  needed  to  initiate  tracks. 

2.  The  time  required  for  the  tracker  to  settle  down  for  straight  flight. 
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4.  The  time  required  for  the  tracker  to  re-establish  straight  flight  steady  state  after 
the  completion  of  the  turn. 

Several  general  observations  are  possible  from  these  plots  concerning  each  manufacturer. 
COTS  A  appears  to  be  tuned  for  straight  flight,  as  its  fusion  velocity  and  position 
estimates  during  turns  are  worse  than  those  for  single  sensor  data.  COTS  B  has  improved 
fusion  performance,  but  being  a  track-to-track  filter  it  requires  a  much  longer  initiation 
period.  COTS  C  has  somewhat  improved  fusion  performance  throughout  relative  to 
mosaic.  COTS  D  has  by  far  the  best  performance  during  turning  flight  periods. 

Because  these  plots  average  absolute  errors,  they  do  not  illustrate  the  possible  oscillatory 
nature  of  the  track  angle  and  groimd  speed  errors,  particularly  during  the  return  to  straight 
flight.  To  illustrate  these  effects,  signed  track  error  and  signed  ground  speed  error  plots 
for  each  of  the  fusion  cases  are  shown  in  Figures  57  through  72,  where  each  dot 
represents  an  actual  data  point  (no  averaging  over  the  tracks). 

These  figures  show  considerable  difference  between  COTS  trackers  in  the  return  to 
straight  flight  behavior  of  the  track  angle  error.  In  particular,  COTS  A  experiences 
significant  overshoot  corrections,  COTS  B  and  C  decay  back  to  truth,  although  COTS  C 
is  considerably  quicker  in  its  performance,  and  COTS  D  shows  oscillation  between 
overshoot  and  undershoot  in  its  return  behavior.  In  the  ground  speed  realm,  only 
COTS  D  maintains  consistently  accurate  estimates  during  and  after  the  turn. 

The  general  conclusion  reached  with  this  test  is  that  multi-sensor  fusion  does  not  appear 
to  provide  sufficient  surveillance  improvements  over  mosaiced  sensor  performance  to 
permit  algorithm  improvements  in  automation  safety  functions,  even  when  biases  are  not 
present  to  distort  the  tracker  performance.  In  fact,  it  appears  to  actually  degrade 
performance  for  some  of  the  COTS  trackers.  Surprisingly,  the  performance  for  the  high 
data  rate  SURVSEF  sensor  is  much  worse  for  some  of  the  trackers;  this  appears  to  be 
due  to  improper  setting  of  the  tracker  parameters  for  4.8-second  updates,  as  closer  noisy 
data  points  can  lead  to  more  apparent  velocity  deviations. 

Only  COTS  tracker  D  appears  to  have  a  turn  state  in  the  Kalman  filter,  with  an  estimate 
of  the  turn  rate.  For  the  other  fusion  trackers,  the  performance  in  turns  consists  only  of 
slowly  adjusting  the  heading  when  undershoot  position  errors  are  first  detected.  Thus,  as 
seen  the  track  angle  reaches  and  maintains  an  approximately  steady  state  error  value 
throughout  the  turn  period.  Some  of  these  trackers  decay  back  toward  true  heading  when 
the  turn  ends,  while  others  tend  to  overshoot  and  oscillate  back  to  truth.  In  all  cases, 
steady  state  requires  a  significant  time  to  re-achieve  when  straight  flight  is  resumed. 

The  tracker  D  actually  computes  a  turn  rate  and  employs  it  in  its  smoothing  and 
prediction  routines.  Figures  73  and  74  present,  for  all  times  and  all  tracks,  the  turn  rate 
values  in  effect.  During  the  actual  3  deg/sec  turn  period,  these  values  appear  to  generally 
be  within  2  deg/sec  of  reality,  although  outliers  exist.  Track  initiation  periods  also  appear 
to  have  turn  rates  error  problems,  and  turn  rates  are  computed  for  some  tracks  at  all  times. 
Thus  although  the  existence  of  a  turn  rate  has  significantly  improved  turning 
performance,  it  does  so  at  the  occasional  cost  of  degraded  straight  flight  accuracy. 
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6.2  BIAS  HANDLING  AND  PERFORMANCE 


System  biases  can  have  a  critical  effect  on  the  efficacy  of  fusion  algorithms.  In  general, 
biases  will  always  affect  the  positional  accuracy  of  predictions.  However,  track  angle 
and  ground  speed  predictions  are  significantly  greater  compromised  by  inter-sensor 
biases  in  report-to-track  fusion  systems  which  enter  all  reports  from  all  sensors  into  the 
same  Kalman  filter  than  in  track-to-track  fusion  systems  which  track  each  sensor's  reports 
separately  and  average  the  results.  For  any  system,  real-time  algorithms  that  attempt  to 
compute  and  eliminate  biases  in  the  input  reports  can  reduce  any  bias  degradation  effects. 

Each  company  fusion  tracker  was  run  three  times  against  the  BasicSim  simulation  data 
base  with  the  following  list  of  input  specifications,  where  double  nominal  biases  mean 
position  and  measurement  biases  set  at  random  for  each  sensor  and  aircraft  to  double  the 
amounts  typically  seen  in  the  real  data; 

1 .  fused  enroute  imbiased  sensors 

2.  fused  enroute  double  nominally  biased  sensors  and  the  initial  bias  file  (i.  e.  no 
bias  corrections  yet  determined) 

3.  fused  enroute  double  nominally  biased  sensors  after  a  1-hour  run  to  allow  the 
bias  file  to  adapt  to  the  biases 

Both  CD2  and  Mode  S  data  accuracies  were  tested.  In  each  case,  the  output  reports  were 
analyzed  for  track  angle,  ground  speed,  and  position  errors  as  a  function  of  time  during 
the  run.  The  results  are  presented  in  Figures  75  through  98. 

As  seen  by  the  results,  all  of  the  COTS  tracker  positions  are  significantly  impacted  by 
biases  in  their  un-adapted  state.  As  expected,  all  the  report-to-track  trackers  in  addition 
introduce  significant  velocity  errors,  while  the  track-to-track  COTS  D  has  its  velocities 
essentially  unaffected  by  biases.  One  interesting  effect  is  seen  in  the  COTS  C  velocity 
results:  biases  appear  to  have  reduced  the  errors,  especially  in  ground  speed,  during  the 
turn.  This  paradox  can  be  explained  by  assuming  that  the  presence  of  biases  has 
prevented  the  filter  gains  from  closing  down  as  far,  so  that  the  filter  is  better  prepared  to 
follow  the  turn. 

Another  test  of  bias  performance  is  to  see  if  the  data  “hops”  noted  in  mosaic  systems 
have  been  ameliorated  by  fusion.  To  this  end,  both  mosaic  and  fusion  runs  were  made 
with  each  COTS  tracker  (except  no  mosaic  run  for  the  track-to-track  COTS  D  for  the 
reasons  expounded  above)  for  the  following  simulation  scenario: 

1.  straight  flight  for  100  seconds,  where  for  each  track  the  initial  preferred  sensor 
is  maintained  for  all  scans,  and  the  same  double  system  biases  assumed  above 
are  present 

2.  removal  of  the  preferred  sensor  reports  for  each  track  starting  at  time  100 

3 .  further  straight  flight  for  anofiier  100  seconds. 

Figures  99  through  122  illustrate  the  effects  of  sensor  switching  in  a  biased  system  by 
presenting  error  statistics  before  and  after  the  preferred  sensor  removal.  Two  points  are 
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clear  from  the  plots.  First,  all  the  COTS  trackers  when  run  in  mosaic  mode  suffered 
significant  disruptions  at  the  time  of  the  preferred  sensor  loss,  with  both  velocity  and 
position  errors  suddenly  jumping.  These  jumps  ranged  from  modest  for  COTS  A  to 
larger  for  COTS  B  to  very  large  for  COTS  C.  Second,  it  is  clear  that  the  significant 
position  and  velocity  hops  seen  for  the  mosaicing  curves  at  time  100  do  not  appear  with 
fusion  for  any  of  the  trackers,  including  COTS  D.  Thus,  the  contention  that  fusion 
provides  seamless  sensor  transition  has  been  validated. 

6.3  SENSOR  OUTAGE 

One  of  the  main  justifications  for  implementing  fusion  is  that  tracking  can  continue 
seamlessly  if  a  sensor  goes  down,  with  other  sensor  reports  covering  for  the  lost  sensor’s 
output.  To  test  the  accuracy  of  tracking  in  such  an  eventuality,  the  BasicSim  scenario 
was  rerun  under  the  assumption  that,  for  each  track,  the  preferred  sensor  was  missing  (the 
anti-mosaic  case). 

Figures  123  through  146  present  the  results  for  this  sensor  outage  case  compared  to  the 
previous  cases  of  mosaic  tracking  and  fusion  tracking.  The  expected  result  was  that  the 
loss  of  the  preferred  sensor  would  degrade  the  fusion  results,  although  not  significantly, 
and  hopefully  that  the  remaining  fusion  would  still  be  as  good  or  better  than  mosaicing. 
In  general,  the  results  indicate  that  the  velocity  errors  (track  angle  and  ground  speed)  are 
affected  to  only  a  minor  degree  by  the  sensor  loss  and  still  remain  superior  to  mosaicing, 
while  the  position  error,  especially  with  CD2  data,  degrades  more  significantly.  The 
position  finding  is  not  a  surprise,  as  the  preferred  sensor  will  always  have  the  least  noisy 
azimuthal  coordinate  (azimuthal  error  is  linear  with  range). 

In  any  event,  the  contention  that  fusion  can  overcome  sensor  outages  and  still  provide 
quality  surveillance  superior  to  mosaicing  is  supported  by  these  results. 

6.4  OUTLIER  RECOVERY 

Any  tracker  will  on  occasion  experience  outlier  input  reports,  either  due  to  noisy  data  or 
to  erroneous  correlations.  Ideally,  the  tracker  will  identify  the  input  to  be  an  outlier, 
rather  than  the  start  of  a  maneuver,  after  the  next  input  is  received.  If  the  tracker  does  not 
have  outlier  recognition  algorithms,  however,  the  tracker  predictions  may  require  several 
scans  to  recover. 

To  test  each  COTS  tracker’s  outlier  resilience,  a  simulation  data  set  was  created  that 
consisted  of  the  following  segments  for  each  of  200  aircraft: 

1.  100  seconds  of  straight  flight  with  unbiased  reports,  each  with  normal  sigma 
noise 

2.  a  single  preferred  sensor  report  with  10-sigma  errors  for  both  range  and  azimuth 

3.  100  seconds  of  additional  straight  flight  reports  as  in  1 . 

These  tests  were  run  for  each  company  for  both  CD2  and  Mode  S  data  quality.  In  each 
case,  both  mosaiced  and  fusion  runs  were  made  for  comparison  purposes. 
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As  before,  the  statistical  analysis  programs  were  used  to  generate  the  absolute  average 
track  angle,  ground  speed,  and  position  errors  for  each  5  second  interval  of  the  run. 
Figures  147  through  170  present  Ae  results  for  each  company  for  the  various  nms 

The  most  important  result  observed  in  these  figures  is  that  although  the  outlier  impacted 
positional  and  velocity  accuracy  at  the  time  of  its  arrival  in  all  cases,  the  fusion  systems 
were  far  less  affected,  and  produced  in  almost  all  cases  far  smaller  errors.  Thus  another 
benefit  of  fusion  over  mosaic  is  seen  to  be  significantly  greater  outlier  resistance. 

The  CD2  runs  experienced  much  greater  outlier  distortion,  as  expected,  as  10-sigma 
errors  are  several  times  greater  in  magnitude  for  CD2  quality  data  than  for  Mode  S  data. 
In  should  be  noted  however  that  all  trackers  accepted  the  bad  data  points,  indicating  that 
their  tracker  correlation  boxes  were  sized  large  enough  to  encompaiss  the  bad  position. 
Thus  error  effects  of  the  magnitude  noted  in  the  figures  could  occur  when  erroneous 
correlations  are  made. 

Recovery  was  achieved  after  a  few  additional  data  points  for  all  companies,  indicating 
tracker  resilience  but  no  outlier  detection  algorithms.  The  level  of  distortion  produced  by 
outliers,  however,  varied  significantly  from  the  best  company  to  the  worst. 

6.5  GENERAL  TRACKING  ACCURACY 

As  described  earlier,  a  RandSim  scenario  was  also  used  to  test  each  COTS  tracker.  This 
scenario  randomly  mixed  straight,  accelerating,  turning,  and  combination  flight  segments 
for  each  track.  The  analysis  computed  the  track  angle,  ground  speed,  and  positional 
errors  for  each  segment  type  as  a  function  of  time  in  the  segment.  To  reduce  the  number 
of  possible  plots  to  a  manageable  size,  we  present  in  Figures  171  through  202  the  average 
absolute  track  angle  and  groxmd  speed  errors  for  several  maneuver  types  for  each  COTS 
tracker  for  both  CD2  and  Mode  S  data  qualities,  assuming  fusion  of  all  enroute  sensors. 
As  is  apparent,  turning  and  accelerations  create  problems  for  all  trackers,  although  the 
degree  of  difficulty  varies  widely. 

The  first  4  figures,  171-174,  present  performance  in  turns  with  CD2  data.  These  results 
confirm  that  only  one  of  the  COTS  trackers,  COTS  C  in  Figure  173,  has  a  turning  state 
with  turn  rate  as  a  variable.  Such  a  state,  when  designed  properly,  correctly  predicts  the 
track  angle  after  the  turn  is  recognized,  reducing  the  error  for  the  remainder  of  the  turn  to 
a  small  value.  The  figures  illustrate  that  for  the  other  companies,  errors  in  the  turns  never 
recover  at  any  time  during  the  turn,  but  experience  significant  undershoot  throughout.  In 
addition,  only  COTS  C  does  not  experience  ground  speed  errors  during  the  turn;  the 
severe  imdershoot  of  the  other  trackers  is  coupled  into  speed  errors  as  well. 

The  next  4  figures,  175-178,  repeat  the  turning  test  for  Mode  S  quality  data.  Although 
this  data  is  several  times  more  accurate,  only  COTS  A  experiences  a  significant 
improvement  in  performance  over  the  CD2  case.  Thus  again,  tuning  issues  are  evident 
for  this  set  of  trackers. 

Linear  acceleration  performance  is  examined  next  in  Figures  179-186.  All  the  trackers, 
as  seen,  experience  ground  speed  errors  that  last  throughout  the  acceleration  period. 
Thus  an  acceleration  state  is  not  present  in  any  of  the  Kalman  filters;  such  a  state  is  far 
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less  useful  in  general  than  a  turning  state.  Note  that  no  track  angle  errors  are  introduced 
in  this  case,  as  expected.  Also,  once  again,  only  COTS  A  is  significantly  improved  with 
Mode  S  data. 

The  next  experiment  combined  turning  and  linear  acceleration,  such  as  might  occur 
during  takeoffs  or  landings.  The  results,  in  Figures  187-194,  show  significant  errors  in 
both  track  angle  and  ground  speed.  In  fact,  the  track  angle  errors  have  been  exacerbated 
by  the  linear  acceleration,  reaching  values  much  higher  than  before.  The  turning  state  of 
COTS  C  has  helped  again  to  reduce  the  errors  in  this  case,  although  the  linear 
acceleration  has  reduced  its  effectiveness. 

The  final  experiment  tested  the  recovery  patterns  of  the  trackers  after  a  maneuver 
terminated;  the  results  are  presented  in  Figures  195-202.  As  seen,  with  CD2  data,  all 
trackers  except  for  COTS  C  required  the  better  part  of  a  minute  to  return  to  normal  error 
performance;  with  COTS  C  the  errors  had  remained  small  during  the  maneuver,  so 
recovery  was  minimal.  With  Mode  S  quality  data,  all  trackers  recovered  significantly 
quicker. 

6.6  TRACK  INITIATION  PERFORMANCE 

A  reasonable  expectation  of  fusion  is  that  by  utilizing  data  from  several  sensors,  tracking 
can  commence  earlier  than  is  possible  with  single  sensor  reports.  This  would  be 
particularly  useful  in  enroute  airspace,  where  sensor  update  rates  are  nominally  12 
seconds.  This  expectation  should  usually  be  true  for  report-to-track  fusion,  where  reports 
from  all  sensors  are  input  to  a  common  tracker.  However,  for  report-to-track  fusion 
trackers  that  initialize  state  vectors  from  single  sensor  reports,  and  for  all  track-to-track 
fusion  trackers,  the  only  possible  gain  occurs  when  a  supplemental  sensor  track  starts 
before  the  preferred  sensor  track. 

Figure  203  presents  the  average  time  required  for  each  COTS  tracker  (plus  the  Lincoln 
adjunct  described  below)  to  first  output  predictions  for  SURVSEF,  mosaic,  and  fusion 
cases  (same  results  assuming  either  CD2  or  Mode  S  quality  data  inputs).  As  expected, 
the  high  data  rate  SURVSEF  sensor  produces  the  fastest  track  initiation.  However,  the 
gain  with  fusion  over  mosaic  was  not  as  dramatic  as  predicted  for  the  report-to-track 
fusion  trackers.  The  longer  initiation  for  vendor  B  (track-to-track)  appears  to  be  an  error 
in  the  tracker;  it  is  being  investigated  by  the  company. 

Of  course  some  COTS  trackers  may  first  output  velocity  later  than  others  to  allow  for 
settling  of  the  estimates.  Thus,  Figure  204  re-plots  the  initiation  times  according  to  when 
the  trackers  first  produce  average  track  angle  errors  of  less  than  10  degrees  in  the 
BasicSim  cases  (initial  straight  flight)  for  CD2  quality  input  data.  As  seen,  these  times 
are  significantly  longer  in  many  cases.  For  Mode  S  quality  data,  the  initial  track 
estimates  were  already  within  10  degrees  of  truth. 

6.7  TRACK  NUMBER  CONSISTENCY 

The  user  of  the  tracker  output  data  will  wish  to  utilize  the  track  number  contained  in  the 
report  to  allocate  the  reports  to  the  aircraft.  Thus,  any  track  number  changes  by  the 
tracker  will  complicate  its  life.  We  observed  numerous  track  number  changes  for  the 
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COTS  trackers  during  our  testing  program.  However,  the  use  of  trial  and  parent/child 
tracking  methods  has  obfuscated  the  actual  number  of  such  changes  that  would  be 
apparent  to  the  user. 

Our  only  concrete  result  is  that  all  COTS  trackers  dropped  some  tracks  in  the  BasicSim 
tests,  most  often  in  the  turning  segment  due  to  loss  of  positional  prediction  accuracy 
caused  by  undershoot  in  the  track  angle  estimate.  Thus,  bad  velocity  estimates,  besides 
leading  to  poor  projections  of  aircraft  positions,  also  lead  to  tracks  disappearing  from  the 
safety  logic  algorithms.  Figure  205  presents  the  percent  of  track  drops  for  each  COTS 
tracker  for  both  CD2  and  Mode  S  quality  data  for  both  mosaic  and  fusion  nms. 

This  figure  indicates  that  5%  track  drop  is  typical  in  the  CD2  fusion  runs  for  all  COTS 
trackers  except  for  one  with  negligible  drops.  This  is  a  disturbing  result,  which  hopefully 
can  be  corrected  by  adjusting  the  tracking  parameter  settings  (although  such  adjustments 
can  degrade  straight  flight  tracking).  All  the  companies  with  CD2  tracking  problems 
perform  significantly  better  with  fusion  than  with  mosaicing,  indicating  that  the  higher 
data  rate  does  indeed  help  prevent  track  drops.  When  Mode  S  data  quality  is  assumed,  all 
trackers  but  one  no  longer  experience  track  drops  with  fusion,  thus  providing  another 
reason  for  the  FAA  to  adopt  Mode  S  data  accuracies  through  ASTERIX  surveillance 
formats. 
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7.  LINCOLN  TRACKER  ROUTINE 

The  above  analysis  has  indicted  minor  surveillance  improvements  at  best  from  data 
fusion.  A  likely  rationale  for  this  is  that  all  the  COTS  fusion  trackers  analyzed  in  this 
study  are  in  essence  extensions  of  single  sensor  trackers.  The  report-to-track  trackers 
input  all  reports  as  received  into  a  common  Kalman  filter;  there  are  just  more  reports  with 
fusion.  The  track-to-track  trackers  use  a  single  sensor  Kalman  filter  for  each  sensor;  the 
outputs  are  then  weighted  averaged. 

When  it  became  clear  that  the  COTS  fusion  trackers  did  not  provide  significant 
improvements  in  the  quality  of  aircraft  state  estimation,  we  wondered  whether  it  would 
be  possible  to  use  imique  attributes  of  multi-sensor  data  to  design  a  tracker  completely 
different  from  the  standard  Kalman  filter.  As  an  example,  Lincoln  developed  a  routine 
that  estimates  velocity,  but  not  position,  for  each  output  report  when  fed  multi-sensor 
data,  and  thus  can  serve  as  an  adjunct  to  a  full  tracker.  This  tracker  routine  was  also 
tested  during  the  study  to  provide  a  benchmark  against  which  to  compare  the  COTS 
performances  achieved. 

The  Lincoln  filter  adjunct  has  the  following  properties: 

1.  It  only  produces  velocity  estimates  -  track  angle  and  ground  speed  -  with  no 
positional  smoothing. 

2.  It  includes  estimates  of  both  turn  rate  and  linear  acceleration. 

3.  It  only  operates  on  tracks  seen  by  2  or  more  sensors. 

4.  It  is  unaffected  by  system  or  transponder  registration  errors  and  biases. 

This  tracker  routine  was  tested  on  the  same  RandSim  simulation  data  sets  as  the  COTS 
trackers,  for  both  CD2  and  Mode  S  fusion  cases.  Figures  206  through  213  present  the 
track  angle  and  ground  speed  errors  for  the  set  of  maneuver  classes  considered  above  for 
the  COTS  trackers. 

As  seen  in  the  results,  this  filter  can  cope  with  all  types  of  maneuvers,  with  negligible 
track  angle  and  grovmd  speed  errors  after  a  short  initial  undershoot.  It  quickly  matches 
both  turn  rates  and  linear  accelerations,  tracking  the  data  throughout  the  maneuver 
duration  in  all  cases.  At  maneuver  termination,  the  filter  experiences  a  short  overshoot, 
with  total  recovery  achieved  in  less  than  30  seconds.  Thus  the  superiority  of  the  velocity 
estimates  of  this  filter  over  those  of  most  of  the  COTS  trackers  when  multi-sensor  data  is 
present,  especially  in  turns  and  accelerations,  is  quite  evident.  The  only  exception  is 
COTS  C,  which  was  able  to  provide  comparable  velocity  estimates  for  non-accelerating 
turns. 

The  accuracy  of  the  turn  rate  calculations  was  also  investigated  by  repeating  the 
BasicSim  test  used  to  study  the  one  COTS  tracker's  turn  rate  values.  Figures  214  and  215 
present  these  results  for  all  tracks  and  all  times  for  both  CD2  and  Mode  S  quality  reports. 
As  seen,  the  turn  rate  values  proceed  to  a  narrow  band  around  the  true  3  deg/sec  rate  early 
in  the  turn,  and  remain  there  until  the  turn  ends,  at  which  time  they  decay  back  to  zero. 
False  turn  rates  during  straight  flight  are  relatively  rare.  It  is  also  clear  that  this  routine  is 
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far  more  accurate  for  Mode  S  data,  indicating  its  ability  to  be  properly  tuned  to  noise 
sigmas  even  in  the  presence  of  system  biases. 

To  be  useful  as  a  stand-alone  tracker,  this  adjunct  must  be  expanded  by  the  addition  of  a 
routine  to  estimate  position  and  a  procedure  to  handle  single-sensor  tracks,  and  outlier 
exception  handlers  must  be  developed. 
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8.  CONCLUSIONS 


The  principal  conclusion  of  this  study  is  that  technology  is  on  the  shelf  (COTS)  to 
achieve  some,  but  not  all,  of  the  surveillance  improvements  that  have  been  promised  with 
sensor  fusion  algorithms. 

The  COTS  products  are  able  to  receive  inputs  from  typical  FAA  radars  and  produce 
output  tracks  that  are  as  good  as  or  better  than  those  produced  by  existing  FAA 
automation  systems.  In  particular,  the  COTS  products  eliminate  the  track  data  “hops” 
and  “stitching”  that  are  characteristic  of  existing  mosaic  systems.  This  could  produce 
substantial  benefits  by  opening  up  the  prospect  of  allowing  controllers  to  use  the  same 
separation  standard  permitted  for  single  sensor  operation  when  multiple  sensors  are  in  use 
in  a  given  sector.  The  COTS  products  are  also  robust  to  the  presence  of  “outlier”  data 
and  die  failure  of  a  single  sensor.  In  addition,  it  may  be  true  that  the  use  of  a  Kalman 
filter  by  itself  in  these  COTS  products  in  either  mosaic  or  fusion  systems  may  lead  to 
significant  surveillance  improvements  over  the  current  alpha-beta  smoothing  (which  was 
not  tested). 

The  results  indicated  that  the  COTS  products  do  not  produce  significantly  better 
estimates  of  position  and  velocity  in  the  fusion  mode  than  in  the  mosaic  mode,  either  in 
straight  line  or  turning  flight.  Intuitively,  one  would  have  expected  the  additional  data 
and  die  associated  higher  update  rate  would  have  contributed  to  improved  performance  in 
these  areas.  Improved  state  estimation  might  produce  benefits  by  allowing  decision 
support  tools  (e.g.,  CTAS,  etc.)  to  better  predict  aircraft  trajectories.  At  the  present  time, 
however,  FAA  requirements  for  aircraft  state  estimation  in  this  context  are  not  well 
developed.  Therefore  it  is  difficult  to  assess  the  impact  of  the  current  set  of  COTS 
products.  An  adjunct  velocity  tracker  may  provide  improved  performance  should  that 
prove  to  be  necessary. 

During  the  various  fusion  tests,  it  became  clear  that  the  performance  in  all  cases  was 
significantly  improved  when  Mode  S  data  quality  (such  as  with  ASTERIX)  was  assumed, 
as  opposed  to  the  CD2  data  format  and  accuracy.  A  particularly  important  consequence 
of  this  improvement  was  that  the  significant  number  of  track  drops  noted  in  aircraft  turns 
with  CD2  data  was  virtually  completely  eliminated  when  Mode  S  quality  data  was  input 
instead  to  the  trackers. 

Our  recommendation  is  that  the  FAA  develop  a  set  of  quantitative  performance  criteria 
that  it  desires  from  a  fusion  tracker.  If  reliability,  robustness,  and  smooth  transitions 
among  sensors  are  the  principal  criteria,  the  current  COTS  products  tested  are  suitable, 
given  appropriate  tuning  and  adaptation.  However,  if  the  desire  is  significant 
improvement  in  aircraft  position  and  velocity  estimation,  additional  development  may  be 
required. 
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igure  1.  Example  oj  Competing  Tracker  Solutions. 


32 


le  8  Minutes  of  Recorded  Data. 
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Mosaic  "Hop"  Smoothing  -  Tracker  versus  Input  Data. 


Scan-to-Scan  Variation  (knots)  Scan-to-Scan  Variation  (deg) 
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Mosaic  CD2  Fused  SURVSEF  Mode  S  Fused 


Figure  10.  Tracker  Track  Angle  Smoothness  versus  Type  of  Data. 


Mosaic  CD2  Fused  SURVSEF  Mode  S  Fused 


Figure  11.  Tracker  Ground  Speed  Smoothness  versus  Type  of  Data. 


38 


Time  (sec) 


360 


Position  Error  (nmi)  Ground  Speed  (knots)  Treck  Angle  (deg) 


Time  (sec) 


Figure  18.  Tracked  Recorded  Data  -  Track  Angle  -  COTS  C. 


Figure  19.  Tracked  Recorded  Data  -  Ground  Speed  -  COTS  C. 


Time  (sec) 


Figure  20.  Tracked  Recorded  Data  -  Position  Error  -  COTS  C. 

41 


Position  Error  (nm)  Ground  Speed  (knots)  Track  Angle  (deg) 
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Figure  24.  Sample  Tracking  Performance  on  Recorded  Data  During  a  Turning  Segment  -  COTS  A. 
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Figure  27.  Sample  Tracking  Performance  on  Recorded  Data  During  a  Turning  Segment  -  COTS  D. 
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Figure  30.  Sample  Printout  from  Merge  Routine  -  Truth  from  Simulation  Input,  Smooth  from  Tracker  Output. 

(Note  All  Inputs  do  not  Generate  an  Output.) 
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Figure  36.  Turning  Test  (T urn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  A. 


Figure  37.  Turning  Test  (T urn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  A. 
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Figure  38.  Turning  Test  (Turn  200-230  Seconds)  -  Position  Error  -  Mode  S  Data  -  COTS  A 
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Figure  42. 


Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle 


Error  -  Mode  S  Data  -  COTS  B 
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Figure  43.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTSB. 


Figure  44.  Turning  Test  (T urn  200-230  Seconds)  -  Position  Error  -  Mode  S  Data  -  COTS  B. 
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Figure  48.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  C. 


Figure  49.  Turning  Test  (Turn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  C. 


Figure  50.  Turning  Test  (Turn  200-230  Seconds)  -  Position  Error  -  Mode  S  Data  -  COTS  C. 
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Figure  51.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  CD2  Data  -  COTS  D. 
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Figure  52.  Turning  Test  (T urn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  D 
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Figure  53.  Turning  Test  (T urn  200-230  Seconds)  -  Position  Error  -  CD2  Data  -  COTS  D. 


Figure  59.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  A 
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Figure  60.  Turning  Test  (T urn  200-230  Seconds)  -  Ground  Speed  Error  -  Mode  S  Data  - 

COTS  A. 
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Figure  61.  Turning  Test  (T urn  200-230  Seconds)  -  Track  Angle  Error  -  CD2  Data  -  COTS  B. 
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Figure  62.  Turning  Test  (T urn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  B. 


6 


Track  Angle  Error  (deg) 


Figure  63.  Turning  Test  (Turn  200-230  Seconds)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  B 
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Figure  66.  Turning  Test  (T urn  200-230  Seconds)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  C. 
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Figure  75.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  CD2  Data  -  COTS  A. 
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Figure  76.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  CD2  Data  -  COTS  A. 
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Figure  77.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  CD2  Data  -  COTS  A. 


Figure  78.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  Mode  S  Data  -  COTS  A. 
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Figure  79.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  Mode  S  Data  -  COTS  A. 


Figure  80.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  Mode  S  Data  -  COTS  A. 
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Figure  81.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  CD2  Data  -  COTS  B. 
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Figure  82.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  CD2  Data  -  COTS  B. 
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Figure  83.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  CD2  Data  -  COTS  B. 
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Figure  84.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  Mode  S  Data  -  COTS  B. 
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Figure  85.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  Mode  S  Data  -  COTS  B. 


Figure  86.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  Mode  S  Data  -  COTS  B. 
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Figure  87.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  CD2  Data  -  COTS  C. 
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Figure  88.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  CD2  Data  -  COTS  C. 
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Figure  89.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  CD2  Data  -  COTS  C. 


72 


Figure  90.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  Mode  S  Data  -COTS  C. 
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Figure  91.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  Mode  S  Data  -COTS  C. 


Figure  92.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  Mode  S  Data  -  COTS  C. 
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Figure  93.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  CD2  Data  -  COTS  D. 
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Figure  94.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  CD2  Data  -  COTS  D. 
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Figure  95.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  CD2  Data  -  COTS  D. 
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Figure  96.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Track 

Angle  Error  -  Mode  S  Data  -  COTS  D. 


Figure  97.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Ground 

Speed  Error  -  Mode  S  Data  -  COTS  D. 


Figure  98.  Bias  Adaptation  Test  (1  Hour  Adaptation  Period)  -  Turn  200-230  Seconds  -  Position 

Error  -  Mode  S  Data  -  COTS  D. 
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Figure  99.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  - 
Track  Angle  Error  -  CD2  Data  -  COTS  A. 
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Figure  100.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  CD2  Data  -  COTS  A. 


Figure  1 01.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  ■ 

CD2  Data  -  COTS  A. 
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Figure  102.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  Mode  S  Data  -  COTS  A. 


Time  (sec) 

Figure  103.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  Mode  S  Data  -  COTS  A. 


Figure  104.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  A. 
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Figure  1 05.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  CD2  Data  -  COTS  B. 
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Figure  1 06.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  CD2  Data  -  COTS  B. 
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Figure  1 07.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

CD2 Data- COTS B. 
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Figure  108.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  Mode  S  Data  -  COTS  B. 
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Figure  109.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  Mode  S  Data  -  COTS  B. 
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Figure  110.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error 

Mode  S  Data  -  COTS  B. 
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Figure  111.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  CD2  Data  -  COTS  C 


Figure  112.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  CD2  Data  -  COTS  C. 


Time  (sec) 

Figure  113.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

CD2  Data  -  COTS  C. 
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Figure  114.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  Mode  S  Data  -  COTS  C. 


Figure  115.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  Mode  S  Data  -  COTS  C. 


Figure  116.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  C. 
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Figure  117.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  CD2  Data  -  COTS  D. 


Figure  118.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  CD2  Data  -  COTS  D. 
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Figure  119.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

CD2  Data  -  COTS  D. 
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Figure  120.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Track  Angle 

Error  -  Mode  S  Data  -  COTS  D. 
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Figure  121.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Ground  Speed 

Error  -  Mode  S  Data  -  COTS  D. 


Figure  122.  Sensor  Hop  Due  to  Biases  Test  -  Sensor  Change  at  100  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  D. 
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Figure  123.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

CD2  Data  -  COTS  A. 


Time  (Sec) 

Figure  124.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

CD2  Data  -  COTS  A. 


Figure  1 25.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

CD2  Data  -  COTS  A. 
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Figure  126.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error 

Mode  S  Data  -  COTS  A. 
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Figure  127.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  A. 


Figure  128.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  A. 
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Figure  129.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

CD2  Data  -  COTS  B. 


Figure  130.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

CD2  Data  -  COTS  B. 


Figure  131.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

CD2  Data  -  COTS  B. 
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Figure  132.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Modes  Data- COTS  B. 


Figure  133.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  B. 


Time  (eec) 

Figure  134.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  B. 
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Figure  135.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error 

CD2  Data  -  COTS  C. 
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Figure  136.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error 

CD2  Data  -  COTS  C. 
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Figure  137.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

CD2  Data  -  COTS  C. 


Figure  138.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Modes  Data- COTS  C. 
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Figure  139.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  C. 


Figure  140.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  C. 
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Figure  144.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Track  Angle  Error  - 

Mode  S  Data  -  COTS  D. 
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Figure  145.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Ground  Speed  Error  - 

Mode  S  Data  -  COTS  D. 


Time  (aae) 

Figure  146.  Preferred  Sensor  Outage  Test  -  Turn  200-230  Seconds  -  Position  Error  - 

Mode  S  Data  -  COTS  D. 
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Figure  147.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  CD2  Data  -  COTS  A. 


Figure  148.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  A. 
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Figure  149.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  CD2  Data  -  COTS  A. 
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Figure  150.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  A. 
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Figure  151.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  A 
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Figure  152.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  A. 
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Figure  153.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  CD2  Data  -  COTS  B. 
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Figure  154.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  B 


Figure  155.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  CD2  Data  -  COTS  B. 
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Figure  156.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  B. 


Figure  157.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  B. 


Figure  158.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  B. 
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Figure  162.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  C. 


Figure  163.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  C 


Figure  164.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  C. 


Figure  1 65.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  CD2  Data  -  COTS  D. 


Figure  166.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  CD2  Data  -  COTS  D. 


Figure  167.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  CD2  Data  -  COTS  D. 
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Figure  168.  Outlier  Test  (Outlier  at  Time  100)  -  Track  Angle  Error  -  Mode  S  Data  -  COTS  D. 


Figure  169.  Outlier  Test  (Outlier  at  Time  100)  -  Ground  Speed  Error  -  Mode  S  Data  -  COTS  D. 


Time  (sec) 


Figure  170.  Outlier  Test  (Outlier  at  Time  100)  -  Position  Error  -  Mode  S  Data  -  COTS  D. 
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Figure  171.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Spet 

CD2  Data  -  COTS  A. 
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Figure  172.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors  - 

CD2 Data- COTS B. 
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Figure  175.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Spec 

Mode  S  Data  -  COTS  A. 


Figure  176.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  COTS  B. 
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Figure  177.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors 

Modes  Data- COTS  C. 


Figure  1 78.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  COTS  D. 
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Figure  1 79.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  CD2  Data  -  COTS  A. 
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Figure  180.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  CD2  Data  -  COTS  B. 
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Figure  181.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  CD2  Data  -  COTS  C. 
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Figure  182.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  CD2  Data  -  COTS  D. 
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Figure  183.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  Mode  S  Data  -  COTS  A. 


Figure  184.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  Mode  S  Data  -  COTS  B. 
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Figure  187.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  CD2  Data  -  COTS  A. 
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Figure  188.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  CD2  Data  -  COTS  B. 
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Figure  189.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  CD2  Data  -  COTS  C. 


Figure  190.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  CD2  Data  -  COTS  D. 
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Figure  191.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  Mode  S  Data  -  COTS  A. 
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Figure  192.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  Mode  S  Data  -  COTS  B. 
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Figure  193.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  Mode  S  Data  -  COTS  C. 
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Figure  195.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Spt 

CD2  Data  -  COTS  A. 
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Figure  196.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors  - 

CD2  Data- COTS  B. 
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Figure  197.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors 

CD2  Data  -  COTS  C. 


Figure  198.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors 

CD2  Data  -  COTS  D. 
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Figure  199.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  COTS  A. 


Figure  200.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  COTS  B. 
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Figure  203.  Time  to  Initiate  Tracking  -  CD2  or  Mode  S  Data. 
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Figure  207.  Maneuver  Test  -  Turning  (After  Straight)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  Lincoln. 
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Figure  208.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  CD2  Data  -  Lincoln. 
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Figure  209.  Maneuver  Test  -  Accelerating  (After  Straight)  -  Track  Angle  &  Ground  Speed 

Errors  -  Mode  S  Data  -  Lincoln. 
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Figure  210.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  CD2  Data  -  Lincoln. 
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Figure  211.  Maneuver  Test  -  Turning  &  Accelerating  (After  Straight)  -  Track  Angle  &  Ground 

Speed  Errors  -  Mode  S  Data  -  Lincoln. 
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Figure  212.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Sj 

CD2  Data  -  Lincoln. 
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Figure  213.  Maneuver  Test  -  Straight  (After  Maneuver)  -  Track  Angle  &  Ground  Speed  Errors 

Mode  S  Data  -  Lincoln. 
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Figure  214.  Computed  Turn  Rate  Values  -  Turn  200-230  Seconds  @  3  Deg/Sec  -  CD2  Data  - 

Lincoln  Filter. 


Figure  215.  Computed  Turn  Rate  Values  -  Turn  200-230  Seconds  @  3  Deg/Sec  -  Mode  S  Data 

Lincoln  Filter. 
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Figure  216.  Sample  Tracking  Performance  on  Recorded  Data  During  a  Turning  Segment  -  Lincoln  Tracker. 
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